Pages

Showing posts with label web services. Show all posts
Showing posts with label web services. Show all posts

Playing with Foursquare API with Python

Wednesday, December 21, 2011

Hi all,

I'd like to share a project that I am developing that it may be useful for anyone who wants to create datasets from mobile location networks.  Specifically, I developed a wrapper in Python for accessing the Foursquare API called PyFoursquare

For anyone who doesn't know what is Foursquare, it is a popular mobile social-location network with more 10.000.000 of users around the world. The idea is that you can share your current location with your friends and as result discover new places, find where your friends are and even check some tips and recommendations about a place and what to do when you arrive there. It is an amazing project with lots of data available for anyone who wants to develop new apps for connect or mine (data mining) its data!

Foursquare Mobile Application

This Python API is one of the results of my master degree project where I proposed a new architecture for mobile recommenders that fetches reviews from social networks to improve the explanation and the quality of the given recommendations.  I  used this library to collect tips (text reviews) from Foursquare from places at my neighborhood Recife, Brazil.  This API was a little messy, so I decided to clean it up, organize and documment it for publish for the open-source community.

One of advantages of this API is that you can handle each entity from the Foursquare data as Model object. So instead of handling with json dictionaries, I encapsulate the results in the respective models (Venue, Tips, User, etc.) and access its attributes as common object in Python!

I inspired myself at the work of Joshua at Tweepy, which is a Python library for Twitter.  In this version released 0.0.1 I only implemented some API's such as search/venues,  venue_details and venue_tips.  In future releases I pretend to add more models and support for more API methods available at Foursquare.

How can you use it at your project ?

It is simple! Just install it by downloading at the Github's home project, extract the source from the tar.gz and  at the directory of the project run the command below:

$ python setup.py install

or the easier way is to install by the command easy_install:

$ easy_install pyfoursquare


After that, you can  simple test by running the command below at your Python Shell

>>> import pyfoursquare


Now let's see how you can get started with the PyFoursquare:

First you need to create an application at Foursquare. The link is this.  There  you can also get further information about the API, another libraries and several applications using the Foursquarw API's.  

The Foursquare Developer's Settings


After creating your application, you must get the client_id and your client_secret. Those keys will be important to connect the app to the users' accounts.  Foursquare uses the secure authentication based on OAuth2.  In PyFoursquareAPI, you won't need to handle with all steps provided by OAuth2.  It already encapsulates all the steps and handshakes between your app and Foursquare servers. \m/ 

Below the  code you must write for authenticate an user to connect to your app:




After the user  authorized, you now can instantiate the PyFoursquare API.  It will give you access to the Foursquare API methods.  I implemented several methods, but feel free to add new ones! Don't forget to submit the final results as pull requests at the project's repository at Github.

In this example I fetched a venue by giving as input the latitude and longitude and querying for the place with the name 'Burburinho'.  Burburinho is a popular bar nearby where I work!

Source code




Now you can access the result and access the Venue as a Python Object. All elements of the Venue are represented as attributes of the object Venue at PyFoursquare. The goal is to make easier the life of the developer when he access the Foursquare API by parsing all the JSON (the result) and placing in the correct model for him.



I expect you enjoyed this API. Feel free to use it at your applications or research!  I'd like to thank the Foursquare team for expose their data by providing those API's!  For data mining researchers instered in mobile location data, it is a mine of gold!

Further information about PyFoursquare, you can find here.

Feel free to give sugestions, improvements and comments,

Regards,

Marcel Caraciolo

Introducing the Architecture REST - Creating APIs - Part 02

Monday, May 16, 2011



In this post, we continue talking about the REST architecture and our progress to create our first API.  This is the second part of the series, if you want to see the Introduction, please check this link

The SOA Principles

The REST is also considered a Service-Oriented Architecture (SOA) , so it the SOA principles must be considered. The REST architecture follows the interoperability by providing support for several types of response. It is a good practice to make your resource accept HTML, JSON or XML in the same method, turning it more portable.  Depending of the target of your application, the number of clients of your services can be bigger in this way. However, providing all those types of response consumes time, and you have to analyze if you are available for working on this interoperability.

It is also important that your RESTful services don't take overloaded operations. This includes heavy processing tasks such as data transformations (XML to JSON, XML to Text/Plain, etc.) as also the validation of the data before each transformation.  This results in the other principle of SOA the weak coupling, that is, make your system less dependent of other modules in order to reduce the modification effects and failure tailoring.  Generally the simple RESTful services systems are divided in the following packages:

  • resources -  The resources of your system which implement the HTTP methods POST, GET, PUT, DELETE. For each request received, it uses the modules of the utils package and the communication provided with dao, if necessary. The resources also have the task of interpret the result , possible failures and response to the client.
  • utils - The utility classes as also related to data transformation.
  • dao -  The classes with the pattern DAO (Data Access Object), responsible for the database transactions
It is also common to see the developers abuse of the methods POST and GET.  Using REST, the developer must know well the four main methods: POST, GET, PUT and DELETE.  We will explain in the next paragraphs about those and comment a little about the HEAD and OPTIONS. For illustrate the explanation, we will use a resource user implemented at http://www.example.com/resources/user/.



POST

Submit the data to be processed at a target resource, which can result in the creation of a new or an update of the existing resources. The data are included in the body of the request. For instance, making a request POST to http://www.example.com/resources/user/  and including the body represented by the JSON in the Figure 01, we will requesting to the server to add this new user. If the user was created successfully, the response will be with a status code 201, pointing that our resource was created.

Figure 1 - JSON as body

GET

Used to request a new representation of the resource specified. In practice, making a request GET in http://www.example.com/resources/user/10 will return as response the user whose the id is 10.  In the other hand if our request was to the url http://www.example.com/resources/user/ , we would have as response the list of all users.

PUT

]It updates the representation of the resource. The data must be sent in the body of the request. Besides, if the URI of the request doesn't not point out to a existing resource, it is allowed to create a new resource with this URI. In our example, if we wanted to update the password of the user with the login "marcelcaraciolo", we just need to make a PUT request to http://www.example.com/resources/user/10 putting the JSON body represented in the next figure, containing the refreshed data.

Figure 2 - JSON as body

DELETE

It deletes a specified resource. It doesn't have the body. If we request the DELETE method to URI http://www.example.com/resources/user/10 it would mean to the serve that we want it to delete the user with the id is equal to 10.

HEAD

Similar to the method GET, but without the body of the response. This method can be used to obtain the metadata of a entity target of the request, without transfer all the data (the body itself of the entity) to the client.

OPTIONS

Return the HTTP methods that the server supports for a specified URI. It can be used to check the features available of a web service. Making a request OPTIONS to http://www.example.com/resources/user/ , we would receive the attribute 'Allow' in the headers with the fields OPTIONS and POST. However making the request OPTIONS to the URI http://www.example.com/resources/user/* ,  we would receive the response OPTIONS, GET, PUT, DELETE and HEAD.  But wait, you may be asking yourself , where is the POST ? Is it missing ?  When you put the wildcat  (*) it is expected some response, but our method POST, using the good practices, is not mapped to accept requests with the URI finishing in 'user/*'.  In other words,  it doesn't make sense to request a POST to 'user/10' , since the id of the resource must be created by the serve. Besides that, in a OPTIONS request, in the body it will come the WADL file, which we will discuss later.

It is important to API developers to know that the client of the service represents the user. So, it is a convention to not use the methods GET and HEAD to do actions,  except the action of information recovering, therefore they are considered safe methods.

Although, knowing that the methods GET and HEAD are secure, the user are conscious about the fact that an action possibly insecure will be requested if it uses the other HTTP methods. To sum up, don't use GET and HEAD to make requests that generate collateral effects. So use the GET method to modify an entity is an anti-pattern and not a good practice in developing REST APIs.

Another important property is the idempotence. PUT, DELETE and OPTIONS are idempotent methods, that is, multiple operations must always return the same result and have the same effect in the application as  one. For instance, making several GET requests to the same URI, it will always return the same response, in case of the requested data didn't change in the interval between the requests.  Finally, the safe methods are indempotent, since it doest not present collateral effects.



HTTP Status Codes

The HTTP Status Codes were created to allow the developers to describe precisely for the clients what happened at the server or even have control of the services. For that, the more specified the response, better. 

The Status Codes has those meanings:

  • 1.xx -  Information
  • 2.xx - Success
  • 3.xx -  Redirect
  • 4.xx -  Client Error
  • 5.xx - Server Error
It is important to developers to know how to response properly with your services. Typically, the responses used by the REST services are 200, 404 or 500, but it is important to understand better the responses of other RESTful services that your application are consuming.  We will see some statuses  codes as follows.

200 - OK
The status code 200 represents that the request was successful. The response returned depends on the method used in the request. If the GET method is used, it will returned the entity corresponding to the requested resource.  In case of POST method, the headers will correspond to the requested resources. Finally if the method was HEAD, it will be returned the headers fields that correspond to the requested resources either.

201 - Created
The request was accomplished and the location of the resource created is returned by the field 'Location' in the response headers. The server must create the resource before return the status code 201. If the resource can't be created immediately, the server must response with 202 (Approved) instead.

202 - Accepted

The request was approved, but it doesn't mean that was finalized. Your purpose is to allow the server to accept asynchronous requests.  Depending on the scenario, it is interesting to include to the body of the response the current status of the request and a path to a state monitor or a time estimative to the request be accomplished.


204 - No Content
The server accomplished the request  successfully but it does not need to response any entity in the body.  Optionally, it may include new or updated meta-information about the the entities in the headers. The response 204 must not have the body. Generally it is the status code response to a DELETE request.

304 - No Modified
If the client has requested with a conditional GET method, but  the documents or the requested data weren't modified, the server must response with this status code. The response 304 must not have a body.

400 -  Invalid Request
Invalid Syntax in the request.

401 -  Not Authorized
The request requires authentication or the user has been refused by the credentials provided.

404 -  Not Found
The server didn't find the resources that correspond to the URI of the request. Generally this response comes as result of a GET request.

409 -  Conflict
There was a conflict in the request with the current state of the resource. This code is only allowed in situations where it expects that the user is able to solve the the conflict and re-send again the request.  For that, the body of the response must include an error message. Generally this scenario occurs in responses to PUT requests.

500 - Internal Server Error
The server came into a unexpected condition that stopped it of finishing the request. For instance, if a problem occurred with the database connection, the response must have the status code 500.


The table below provided gives you a resume of the most used http statuses codes when you are developing your RESTful services.  Please read carefully and know how to use it correctly in order to help the developers that will build applications that will consume your services and know how to handle properly the responses of your API.

HTTP protocol version 1.1 Server Response Codes

In the next post about the REST Services I will present a new service that I am developing using the concepts explained in this series of posts.  It will be related to Location + Question and Answers + Mobile + Python with real code of course! 

I hope you enjoyed,

Marcel Caraciolo

References

[2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html