| View previous topic :: View next topic |
| Author |
Message |
midnite
Guru
Joined: 09 Apr 2006
Posts: 451
Location: Hong Kong
|
 Posted: Mon Jun 02, 2008 12:35 pm Post subject: [SOLVED] PHP HTTP Functions / fsockopen |
 |
|
i am not sure if i should post this thread here. But i am sure that some of the members here runs a PHP server would come across my problem and probably have some solutions 
i run Apache, (on Gentoo, of course).
Very simple, i want to use PHP HTTP functions (http://hk2.php.net/manual/en/ref.http.php).
Why? i want to use PHP to login and fill in a form. Probably http_post_data() is the best choice. Any other solutions?
Yet i have the following error when i access the PHP page: | Quote: | | Fatal error: Call to undefined function http_get() in /var/www/localhost/htdocs/testing.php on line 9 |
i have studied the PHP HTTP function installation & configuration and i guess, i still have not install that plugin. So i checked the USE flags: | Code: | # emerge -pv php
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] dev-lang/php-5.2.6_rc4 USE="apache2 berkdb bzip2 cgi cjk crypt ctype curl curlwrappers exif force-cgi-redirect ftp imap ipv6 ldap ldap-sasl mysql mysqli nls pic readline session sharedext sharedmem simplexml snmp sockets spell spl ssl tidy tokenizer truetype unicode xml xmlreader xmlwriter zip zlib -adabas -bcmath -birdstep -calendar -cdb -cli -concurrentmodphp -db2 -dbase -dbmaker -debug -discard-path -doc -empress -empress-bcs -esoob -fastbuild -fdftk -filter -firebird -flatfile -frontbase -gd -gd-external -gdbm -gmp -hash -iconv -inifile -interbase -iodbc (-java-external) -json -kerberos -libedit -mcve -mhash -msql -mssql -ncurses -oci8 (-oci8-instant-client) -odbc -pcntl -pcre -pdo -posix -postgres -qdbm -recode -reflection -sapdb -soap -solid -sqlite -suhosin -sybase -sybase-ct -sysvipc -threads -wddx -xmlrpc -xpm -xsl -yaz -zip-external" 0 kB
Total: 1 package (1 reinstall), Size of downloads: 0 kB |
i really have no clue what is the next step i should do 
_________________
- midnite.
Last edited by midnite on Thu Jun 05, 2008 3:36 pm; edited 3 times in total |
|
| Back to top |
|
 |
Desintegr
l33t
Joined: 25 Mar 2004
Posts: 863
Location: France - Orléans
|
| Posted: Mon Jun 02, 2008 3:11 pm Post subject: |
|
|
You need dev-php5/pecl-http.
_________________
Gentoo ~AMD64
Hoc Volo, Sic Jubeo !
Mon wiki : http://desintegr.free.fr |
|
| Back to top |
|
|
midnite
Guru
Joined: 09 Apr 2006
Posts: 451
Location: Hong Kong
|
| Posted: Mon Jun 02, 2008 3:29 pm Post subject: |
|
|
| Desintegr wrote: | | You need dev-php5/pecl-http. |
thanks!! 
sorted 
_________________
- midnite. |
|
| Back to top |
|
|
midnite
Guru
Joined: 09 Apr 2006
Posts: 451
Location: Hong Kong
|
| Posted: Tue Jun 03, 2008 4:18 am Post subject: |
|
|
one more question, can they manage HTTPS connections?
_________________
- midnite. |
|
| Back to top |
|
|
Erulabs
n00b
Joined: 08 Mar 2006
Posts: 48
Location: erulabs.com
|
| Posted: Tue Jun 03, 2008 11:07 pm Post subject: |
|
|
| Quote: | | one more question, can they manage HTTPS connections? |
As far as I know dev-php5/pecl-http cannot.
You can do it the manual way and hack together some sockets.
With sockets you'd do something like
| Code: | | $fp = fsockopen("ssl://127.0.0.1", 443, $errno, $errstr); |
See http://us2.php.net/function.fsockopen.
Hope this helps! Sockets rule by the way. Its more than possible to write an entire webserver app in pure PHP using sockets in under 20 lines of code. (Who said PHP wasn't concise?)
_________________
No more things should be presumed to exist than are absolutely necessary. Or really cool. |
|
| Back to top |
|
|
midnite
Guru
Joined: 09 Apr 2006
Posts: 451
Location: Hong Kong
|
| Posted: Wed Jun 04, 2008 9:11 pm Post subject: |
|
|
| Erulabs wrote: | | Quote: | | one more question, can they manage HTTPS connections? |
As far as I know dev-php5/pecl-http cannot.
You can do it the manual way and hack together some sockets.
With sockets you'd do something like
| Code: | | $fp = fsockopen("ssl://127.0.0.1", 443, $errno, $errstr); |
See http://us2.php.net/function.fsockopen.
Hope this helps! Sockets rule by the way. Its more than possible to write an entire webserver app in pure PHP using sockets in under 20 lines of code. (Who said PHP wasn't concise?) |
Thanks Erulabs.
If it cannot handle HTTPS, it will throw an error? or else?
It is strange that (with pecl-http functions by GET or POST) i seem getting the correct page back. But when i check in Firefox Tamper Data, i should have a SESSID cookie. But i cannot find it in the string returned by pecl-http function. =(
If pecl-http function works, it will be very fine. i will try your socket approach later but i am not familiar with sockets.
Thanks =)
_________________
- midnite. |
|
| Back to top |
|
|
midnite
Guru
Joined: 09 Apr 2006
Posts: 451
Location: Hong Kong
|
| Posted: Thu Jun 05, 2008 3:35 pm Post subject: |
|
|
Hi Erulabs,
Thanks very much!!!! Really a million THANKS!!!!
i am now working on sockets. i am now halfway success. i have succeeded logged in and accessing forms by forms in my account!! And the most thrilling point is, i can see the future. i can see i am very near to success. i believe there will be no more difficulties once i manage to submit one form, i can submit as many forms as i want!!
Really thanks a lot!!
cheers =D
_________________
- midnite. |
|
| Back to top |
|
|
midnite
Guru
Joined: 09 Apr 2006
Posts: 451
Location: Hong Kong
|
| Posted: Fri Jun 06, 2008 1:49 am Post subject: |
|
|
i have succeed my whole project!! COOL!!
but one thing i would like to say, fsockopen then fgets is much slower than file_get_contents or http_get. (Or i don't know how to use?)
_________________
- midnite. |
|
| Back to top |
|
|
jure1873
Apprentice
Joined: 09 Feb 2006
Posts: 183
|
| Posted: Fri Jun 06, 2008 2:49 pm Post subject: |
|
|
| I've triedn doing this with php but it's easier with wget - you can tell it to save cookies in a file, post form fields, etc... if you're trying something easy like just submitting a form you can also try wget |
|
| Back to top |
|
|
Erulabs
n00b
Joined: 08 Mar 2006
Posts: 48
Location: erulabs.com
|
| Posted: Fri Jun 06, 2008 11:42 pm Post subject: |
|
|
midnite:
Yeah, sockets are very cool. I would guess your problem with sockets taking longer has to do with the remote server. Try sending "Connection: close" in your headers.
Heres a demo you might find interesting:
| Code: | $ip = "127.0.0.1";
$port = "443";
$post = "this=that&thisotherthing=thattoo\n";
$head .= "POST /path/to/file/to/post/to HTTP/1.1\n";
$head .= "Host: who.were.talking.to\n";
$head .= "Connection: close\n";
$head .= "User-Agent: PHPscript\n";
$head .= "Cookie: acceptCookies=true;\n";
$head .= "Content-Type: application/x-www-form-urlencoded\n";
$head .= "Content-Length: ".strlen($post)."\n\n";
$head .= $post;
$open = fsockopen("ssl://".$ip,$port);
fputs($open, $head);
stream_set_blocking($open, TRUE);
stream_set_timeout($open,7);
$info = stream_get_meta_data($open);
$read = fread($open,209);
eregi("Set-Cookie:.*", $read, $setcookie);
$session = substr($setcookie[0], 23, 32);
while ((!feof($open)) AND (!$info['timed_out'])) {
$stats .= fgets($open, 1496);
$info = stream_get_meta_data($open);
flush();
}
|
This allows you to post information to an ssl (https) protected server, and it does it fast =D
Note here that $session is going to be your JSESSION id. You'll probably need to modify this if you're talking to anything but the lame JSP server I coded this for.
Also, $read is the HTTP headers the server responds with (I only needed the first 209 bytes).
Another bit of advice, check out https://addons.mozilla.org/en-US/firefox/addon/3829 for all your http header capturing needs
_________________
No more things should be presumed to exist than are absolutely necessary. Or really cool. |
|
| Back to top |
|
|
midnite
Guru
Joined: 09 Apr 2006
Posts: 451
Location: Hong Kong
|
| Posted: Sat Jun 07, 2008 9:40 am Post subject: |
|
|
Thanks Erulabs!
i would like to ask what is the different between "Connection: close" and "Connection: keep-alive"? i used the Firefox Tamper Data plugin and found that it is sending "Connection: keep-alive". So i put "Connection: keep-alive" in my PHP scripts to simulate the POST request. Is it one of the point that makes my script slow?
And also, why you know you need only 209 bytes? Is it a standard length of the response header? Or it just apply to your server?
i see you use "flush()". Will it be faster? Or slower as it perform one more action?
Here's my script: | Code: | define ('debug', false);
define ('sslSRC_host', 'www.domain.com');
define ('sslSRC_action', '/login_page');
define ('sslSRC_term', '/index_page');
function addout (&$o) {
$o .= 'Host: '.sslSRC_host."\r\n";
$o .= 'User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14'."\r\n";
$o .= 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'."\r\n";
$o .= 'Accept-Language: en-gb'."\r\n";
$o .= 'Accept-Encoding: gzip,deflate'."\r\n";
$o .= 'Accept-Charset: UTF-8,*'."\r\n";
$o .= 'Keep-Alive: 300'."\r\n";
$o .= 'Connection: keep-alive'."\r\n";
}
function getSESSID (&$p) {
while (!feof($p)) {
$ws = fgets($p, 128);
if ($pos = stripos ($ws, 'SESSID=')) return substr ($ws, $pos, stripos ($ws, "\r\n")-$pos);
}
return false;
}
$POST_DATA = 'data1=one&data2=two';
// submit login (socket)
echo '<h1>submit login (socket)</h1>';
$out = 'POST '.sslSRC_action." HTTP/1.1\r\n";
addout ($out);
$out .= 'Referer: https://www.domain.com/login_page'."\r\n";
$out .= 'Content-Type: application/x-www-form-urlencoded'."\r\n";
$out .= 'Content-Length: '.strlen ($POST_DATA)."\r\n\r\n";
$out .= $POST_DATA."\r\n";
if (debug) echo $out.'<hr />';
if (!$fp = fsockopen ('ssl://'.sslSRC_host, 443)) exit ('fsockopen failed');
fwrite($fp, $out);
if (!$SESSID = getSESSID ($fp)) exit ('getSESSID failed');
if (debug) echo $SESSID.'<hr />';
fclose ($fp);
// submit term (socket)
echo '<h1>submit term (socket)</h1>';
$out = 'POST '.sslSRC_term." HTTP/1.1\r\n";
addout ($out);
$out .= 'Referer: https://www.domain.com/index_page'."\r\n";
$out .= 'Cookie: '.$SESSID."\r\n";
$out .= 'Content-Type: application/x-www-form-urlencoded'."\r\n";
$out .= 'Content-Length: 14'."\r\n\r\n";
$out .= 'term_in=200806'."\r\n";
if (debug) echo $out.'<hr />';
if (!$fp = fsockopen ('ssl://'.sslSRC_host, 443)) exit ('fsockopen failed');
fwrite($fp, $out);
if (!$SESSID = getSESSID ($fp)) exit ('getSESSID failed');
if (debug) { echo $SESSID.'<hr />'; printTable ($fp, 'SUMMARY="Current Schedule"'); echo '<hr />'; }
fclose ($fp);
...
... |
i have several forms to submit. As you can see above, i have to close the first connection and reopen another connection again. Is there a way not to close and reopen again?
Thanks =)
_________________
- midnite. |
|
| Back to top |
|
|
Erulabs
n00b
Joined: 08 Mar 2006
Posts: 48
Location: erulabs.com
|
| Posted: Sat Jun 07, 2008 9:40 pm Post subject: |
|
|
midnite:
You don't need to fclose() and then fsockopen() again.. you can just keep one connection going the entire time.
Also, the difference between "Connection: close" and "Connection: keep-alive" has to do with how the browser and web-server communicate. Things like active DOM, AJAX, and other technologies need a "keep-alive" connection to perform. Browsers like Mozilla FireFox wait until they receive an end of data command from the server, rather than just asking for a truck load of data. In the case of a PHP socket, a "keep-alive" request is going to be 99.9% unnecessary. There are a few complex actions you can take in terms of CGI communications with the server - I would doubt you'll need them.
As for the 209 bytes, no, thats no standard. I used that because thats everything up to the html data in the script I was using. You'll probably find you wont need more than 300 bytes of HTTP headers.
flush() "Flushes the output buffers of PHP and whatever backend PHP is using (CGI, a web server, etc). This effectively tries to push all the output so far to the user's browser.".
This is used in congress with "connection: close" to get the data from the webserver as quickly as possible.
Also, the use of stream_get_meta_data is used to determine if the connection has timed out while we are fetching data from it (as to prevent our script from needing to timeout on its own)
| Code: | while ((!feof($open)) AND (!$info['timed_out'])) {
$stats .= fgets($open, 1496);
$info = stream_get_meta_data($open);
flush();
} |
One other note, "\r" is evil. I can almost promise you'll have an easier time skipping them all together. Try to follow a strict standard for HTTP requests:
GET/POST /what HTTP/version {LINEBREAK '\n'}
HOST: Who.were.talking.to {LINEBREAK '\n'}
User-Agent: MyFakeBrowser {LINEBREAK '\n'}
Content-Length: LENGTH OF POST DATA {LINEBREAK '\n'}
{LINEBREAK '\n'}
POST DATA {LINEBREAK '\n'}
_________________
No more things should be presumed to exist than are absolutely necessary. Or really cool. |
|
| Back to top |
|
|
midnite
Guru
Joined: 09 Apr 2006
Posts: 451
Location: Hong Kong
|
| Posted: Sun Jun 08, 2008 6:56 pm Post subject: |
|
|
Thanks Erulabs for your detailed explanation =)
i also believe that i don't need to fclose() and fsockopen() again. But i got something interesting. i randomly found a simple webpage for demo: www.something.com
Here's my simple code: | Code: | <?php
function sendheader () {
$out = 'GET / HTTP/1.1'."\r\n";
$out .= 'Host: www.something.com'."\r\n";
$out .= 'User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14'."\r\n";
$out .= 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'."\r\n";
$out .= 'Accept-Language: en-gb'."\r\n";
$out .= 'Accept-Encoding: gzip,deflate'."\r\n";
$out .= 'Accept-Charset: UTF-8,*'."\r\n";
$out .= 'Keep-Alive: 300'."\r\n";
$out .= 'Connection: keep-alive'."\r\n\r\n";
return $out;
}
// first time
echo '<h1>first time</h1>';
if (!$fp = fsockopen ('www.something.com', 80)) exit ('fsockopen failed');
fwrite($fp, sendheader ());
while (!feof($fp))
echo fgets ($fp);
// again
echo '<h1>again</h1>';
// without fsockopen here
fwrite($fp, sendheader ());
while (!feof($fp))
echo fgets ($fp);
fclose ($fp);
?> |
and this what i get: | Quote: | <h1>first time</h1>HTTP/1.1 200 OK
Date: Sun, 08 Jun 2008 18:06:48 GMT
Server: Apache/1.3.34 (Unix)
Set-Cookie: c1=1212948408; path=/
Set-Cookie: c2=1; path=/
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=ISO-8859-1
4e
<html><head><title>Something.</title></head>
<body>Something.
</body>
</html>
0
<h1>again</h1> |
Note that the second response is lost.
But if i alter my code a bit: | Code: | <?php
function sendheader () {
$out = 'GET / HTTP/1.1'."\r\n";
$out .= 'Host: www.something.com'."\r\n";
$out .= 'User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14'."\r\n";
$out .= 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'."\r\n";
$out .= 'Accept-Language: en-gb'."\r\n";
$out .= 'Accept-Encoding: gzip,deflate'."\r\n";
$out .= 'Accept-Charset: UTF-8,*'."\r\n";
$out .= 'Keep-Alive: 300'."\r\n";
$out .= 'Connection: keep-alive'."\r\n\r\n";
return $out;
}
// first time
echo '<h1>first time</h1>';
if (!$fp = fsockopen ('www.something.com', 80)) exit ('fsockopen failed');
fwrite($fp, sendheader ());
for ($i=0; $i<5; $i++) // < ---------- changed here ONLY
echo fgets ($fp);
// again
echo '<h1>again</h1>';
// without fsockopen here
fwrite($fp, sendheader ());
while (!feof($fp))
echo fgets ($fp);
fclose ($fp);
?> |
Then the second response will be append to the end of the first response: | Quote: | <h1>first time</h1>HTTP/1.1 200 OK
Date: Sun, 08 Jun 2008 18:29:32 GMT
Server: Apache/1.3.34 (Unix)
Set-Cookie: c1=1212949772; path=/
Set-Cookie: c2=1; path=/
<h1>again</h1>Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=ISO-8859-1
4e
<html><head><title>Something.</title></head>
<body>Something.
</body>
</html>
0
HTTP/1.1 200 OK
Date: Sun, 08 Jun 2008 18:29:33 GMT
Server: Apache/1.3.34 (Unix)
Set-Cookie: c1=1212949773; path=/
Set-Cookie: c2=1; path=/
Keep-Alive: timeout=15, max=99
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=ISO-8859-1
4e
<html><head><title>Something.</title></head>
<body>Something.
</body>
</html>
0
|
Note that there are TWO responses.
So, i have a conclusion that if i did not exhauste the file pointer $fp to end-of-file, it will append. Otherwise, not.
But why?
And i found a different behavior between "Connection: close" and "Connection: keep-alive": | Code: | <?php
function sendheader () {
$out = 'GET / HTTP/1.1'."\r\n";
$out .= 'Host: www.something.com'."\r\n";
$out .= 'User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14'."\r\n";
$out .= 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'."\r\n";
$out .= 'Accept-Language: en-gb'."\r\n";
$out .= 'Accept-Encoding: gzip,deflate'."\r\n";
$out .= 'Accept-Charset: UTF-8,*'."\r\n";
$out .= 'Keep-Alive: 300'."\r\n";
$out .= 'Connection: close'."\r\n\r\n"; // < -------------- changed here ONLY
return $out;
}
// first time
echo '<h1>first time</h1>';
if (!$fp = fsockopen ('www.something.com', 80)) exit ('fsockopen failed');
fwrite($fp, sendheader ());
for ($i=0; $i<5; $i++) // < ---------- changed here ONLY
echo fgets ($fp);
// again
echo '<h1>again</h1>';
// without fsockopen here
fwrite($fp, sendheader ());
while (!feof($fp))
echo fgets ($fp);
fclose ($fp);
?> |
Then the second response will be lost again: | Quote: | <h1>first time</h1>HTTP/1.1 200 OK
Date: Sun, 08 Jun 2008 18:38:19 GMT
Server: Apache/1.3.34 (Unix)
Set-Cookie: c1=1212950300; path=/
Set-Cookie: c2=1; path=/
<h1>again</h1>Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=ISO-8859-1
4e
<html><head><title>Something.</title></head>
<body>Something.
</body>
</html>
0
|
There is really a lot to learn about HTTP connections :-)
And one more question: what does the 4e and 0 mean? i guess 4e should be a hex. It means the content has 78 bytes in total?
Is it true that if i use connection: close, i have to check $info['timed_out']? So i will risk connection timeout and lead to my process failure. If yes, i think i will stick to connection: keep-alive. Slower, but i will have better guarantee to success =)
Why "\r" is evil? i use \r\n because i found the response header uses \r\n too. But anyway, i believe you and try to follow your method. Yet, does HOST: Who.were.talking.to {LINEBREAK '\n'} means echo 'HOST: Who.were.talking.to {LINEBREAK \'\n\'}'; or echo 'HOST: Who.were.talking.to '.{LINEBREAK '\n'};?
Lots of questions, sorry =P
Thanks a lot, again =)
_________________
- midnite. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
