Using cURL With PHP To Fetch A Webpage

by Paul Robinson on 02.08.2010 | no comments

With APIs such as Twitter’s becoming more & more popular connecting to a webservice or webpage in a script is now a common request. As such here is how to fetch a webpage via PHP using cURL.

To fetch a webpage or results from a webservice such as an API using cURL is actually simpler and most people think. Let’s get straight into it.

//initialize a new curl resource
$ch = curl_init();
//Fetch the timeline
curl_setopt($ch, CURLOPT_URL, 'https://example.com/');
//send data via $_GET
curl_setopt($ch, CURLOPT_GET, 1);
//do not return the header information
curl_setopt($ch, CURLOPT_HEADER, 0);

	//If authentication is needed
	curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
	//Set user and pass
	curl_setopt($ch, CURLOPT_USERPWD, "user:pass");

//If SSL verification is needed. Delete if not needed
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, TRUE);
//Give me the data back as a string... Don't echo it.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//Warp 9, Engage!
$content = curl_exec($ch);
//Close CURL connection & free the used memory.
curl_close($ch);

The code here is actually more complicated than normal. However I thought I would give a few extra cURL features that generally aren’t covered.

Let’s go through a few of the more complicated lines and explain what they do.

curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt($ch, CURLOPT_USERPWD, "user:pass");

Should the server you are connecting to require a login, you can use these lines to set the username & password. If you do not need to login to the server, you can delete them.

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, TRUE);

If you are connecting to a https:// connection you can choose whether to verify the SSL certificate of the server. Again if you do not need this or you aren't connecting to a https:// connection you can delete this line.