Pages

Recommender Engines and Data mining with Python at PythonBrasil Conference

Thursday, October 28, 2010

Hi all,

Last week I was at an important event in Curitiba called Python Brasil. It is a annual event where it joins several brazilian developers to discuss about technology and of course about the programming language Python.

I also had the opportunity  to lecture three presentations about several topics of my interest.   The official presentation was about recommendation engines with Python.  This work shows how developers could use python in developing recommendation engines with several examples and explaining the main concepts behind this subject.  The best part was the demos where I used real data from the web such as Twitter, to suggest users that are similar to me among the PythonBrasil followers.  The other example is based on collective buying, which i crawled some popular brazilian web sites and gathered real offers at Curitiba. The main idea is to recommend new offers according to my interests and what people similar to me also liked. It is the classical example of the collaborative filtering, commonly used at several e-commerces today including Amazon.
The presentation was great with lots of feedback.  If you want to take a look at the slides (it is on portuguese) please take a look here:







The main contribution of this work is a new library for building recommendation engines in Python language called Crab.  I've decided with some colleagues to develop this library in order to be a powerful tool for developers to use python as the main language to build and use classical recommender algorithms in their applications. Besides it is extensible so developers can add easily new algorithms to the engine. There is also a easy API so users can plug with their web apps running in different platforms such as Django, AppEngine, Web2Py, etc. 

If you want to collaborate or interest about this subject please feel free to join us at this work. The project is hosted at GitHub with the link:



My second presentation was a lighting talk. What's that ?  There is a extra category of presentations in PythonBrasil where you have 5 minutes to speak about any topic you want.  The one rule is 5 minutes, no more!   I was challenged so I one day before the presentation to develop a web crawler in order to scrap all the lectures submitted and approved at PythonBrasil conference.  With all this data in my hands I've decided to make an analysis to answer three questions in my mind:

a) Which are the main and  frequent topics showed at the keynotes at Python Brazil ?

b) Based on this information, how we could organize the speakers based on those topics ? That is group speakers with similar topics. A classical problem of clustering.

c) What information we could also extract such as level of expertise of the lectures, total time spent in the lectures, etc.


For all those questions I was seeking to answer I decided to use Python, Matplotlib and Ubigraph (A 3D Visualization tool for graphs).   It was really interesting because I really could find some groups based on similar interests.  The main subjects was Entrepreneurship, Hardware, Web, Design Patterns, Data Mining, Django and Artificial Intelligence.

With those subjects I could now group the speakers using a simple clustering algorithm such as K-means and organize them based what their topics were. I've recorded a simple video to present the result using the tool Ubigraph. Take a look:







The presentation in portuguese  you can see here:





In the end I think the event was awesome some great keynotes and of course lots of new contacts at my network.  I have to say it is a great opportunity to meet great people and share ideas and technology!

Next year it will be in São Paulo, Brazil. I expect to be there !

Best regards,

Marcel Caraciolo

5 comments:

  1. This professional hacker is absolutely reliable and I strongly recommend him for any type of hack you require. I know this because I have hired him severally for various hacks and he has never disappointed me nor any of my friends who have hired him too, he can help you with any of the following hacks:

    -Phone hacks (remotely)
    -Credit repair
    -Bitcoin recovery (any cryptocurrency)
    -Make money from home (USA only)
    -Social media hacks
    -Website hacks
    -Erase criminal records (USA & Canada only)
    -Grade change
    -funds recovery

    Email: onlineghosthacker247@ gmail .com

    ReplyDelete
  2. 스포츠중계 A good web site with interesting content, this is what I need. Thanks for keeping this web site, I will be visiting it.

    ReplyDelete
  3. 안전카지노사이트 Thanks for one’s marvelous posting! I seriously enjoyed reading it.


    ReplyDelete
  4. Good blog. Keep sharing. I love them Are you also searching for Cheap assignments? we are the best solution for you. We are best known for delivering writing services to students without having to break the bank

    ReplyDelete