Pages

High Performance Computation with Python - Part 04

Monday, October 10, 2011


Hi all,

This article is the fourth one of the series about High Computation with Python.  For anyone that missed the first, second and third parts check this link about Python Profiling, this one about Cython and finally this about  Numpy Vectors. The goal is to present approaches to make CPU-demanding tasks in Python run much faster.

The techniques that are being covered:

  1.  Python Profiling - How to find bottlenecks
  2.  Cython -  Annotate your code and compile to C
  3.  Numpy Vectors - Fast vector operations using numpy arrays
  4.  Numpy integration with Cython - fast numerical Python library wrapped by Cython
  5.  PyPy - Python's new Just in Time  Compiler
In this post I will talk about PyPy - the JIT Compiler for Python!

The Problem


In this series we will analyze how to optimize the statistical Spearman Rank's Correlation coefficient,  which it is a particular measure used to compute the similarity between items in recommender systems and assesses how well the relationship between two variables can be described using a monotonic function. The source code for this metric can be found in the first post.


Pypy

PyPy is a Just in TIme compiler for the Python programming language. It is multi-platform and it runs Python 2.7.  With your code running in PyPy, it will make your code (depending on how you write your code) run faster (2 - 10 x speed-ups).   Sometimes some work has to be done in the code because of the use of shortcuts that works in CPython that aren't actually correct in the Python specification.

You can download and install PyPy here. To install it, just place it in your home directory and put a symlink from somewhere to it.  Let's run the spearman.py with PyPy and without Python and see the performance difference;



The difference is about 34.77% faster with PyPy against pure Python  considering the input with 190340 on my Macbook. The amazing part is that I didn't change any line of my code! \m/  
If you aren't using a C library like numpy then you should check PyPy - it might just make your code run several times faster. They are still porting PyPy to support Numpy since it some C libraries required   for running Numpy must be rewritten.   You can see some benchmarks of the porting here.

Although the PyPy team gave us a simple integration with array interfaces that behaves in a numpy-like fashion,  for now it has very few functions and only supports double arithmetic.



I strongly recommend you to take a look at PyPy, it shows a great promise for high performance Python with little effort and specially for the scientific community with the support with existing numpy would be a great advance!

I didn't mention until now by I will write a special post to close this series with High Performance with Python: It is about the module multiprocessing and how you can work with it. I will show some examples and a library called JobLib that wraps it where you can easily use the power of the processors of your machine and do some parallell work.


See you next time,

Regards,

Marcel Caraciolo



26 comments:

  1. you might have an excellent weblog right here! would you like to make some invite posts on my weblog?

    ReplyDelete
    Replies
    1. Python is the programming language and its next level and high professional languages here they can create software or programs. Side by side I'm also doing online test takers to use my free time.

      Delete
  2. Thanks for such an interesting article here. I was searching for something like that for quite a long time and at last I have found it here. best placement engineering college in punjab

    ReplyDelete
  3. Great post. Thank you for sharing such useful information. Please keep sharing

    Best B.Tech College in Noida

    ReplyDelete
  4. This professional hacker is absolutely reliable and I strongly recommend him for any type of hack you require. I know this because I have hired him severally for various hacks and he has never disappointed me nor any of my friends who have hired him too, he can help you with any of the following hacks:

    -Phone hacks (remotely)
    -Credit repair
    -Bitcoin recovery (any cryptocurrency)
    -Make money from home (USA only)
    -Social media hacks
    -Website hacks
    -Erase criminal records (USA & Canada only)
    -Grade change
    -funds recovery

    Email: onlineghosthacker247@ gmail .com

    ReplyDelete
  5. We hire only profoundly qualified and subject-explicit experienced assignment writers who are knowledgeable with college schedule and guidelines guaranteeing a legitimate and exact comprehension of your singular necessities and go about as the most suitable objective for "do assignment for me" search. write my assignment

    ReplyDelete
  6. Millions of students have achieved success with online Assignment Help because it has helped them to get whatever they want. Most of the students of Qatar are successful because the assignment writing help has worked the best for them. You should also invest in some good quality assignment writing assistance if you cannot write your assignment by yourself.

    ReplyDelete
  7. Studyprovider has experts team are giving the homework help, assignment help, report, thesis, research writing services and pshchology assignment help available 24/7 seven days a week contact now.

    ReplyDelete
  8. Studyprovider has experts team are giving the homework help, assignment help, report, thesis, research writing services and business law help available 24/7 seven days a week contact now.

    ReplyDelete
  9. Great bloog here! Also your website loads up very fast! What web host are you using? Can I get your affiliate link to your host? I wih my website loaded up as quickly as yours
    경마
    온라인경마

    ReplyDelete
  10. bhunamanasana benefits is a Hatha Yoga Pose that has been practiced for centuries. It's also known as spinal twist prostration pose.

    ReplyDelete
  11. A panther and a hawk are the two true spirit animals for the Gemini Zodiac Sign. cancer spirit animal (June 21st to July 22nd). Cancer-borns are much more.

    ReplyDelete
  12. Marketing Assignment Help Uk provides you with the assignment services because marketing assignment is the process of developing, planning, strategies, analysis, and implementation of programs. Our assignment helpers are highly experienced in their own field. They are highly efficient in the field. So hurry up and visit our website.

    ReplyDelete

  13. Wish to see much more like this. Thanks for sharing your information.

    ReplyDelete
  14. Your blog got me to learn a lot, thanks for sharing.

    ReplyDelete

  15. Thank you so much for letting me express my feeling about your post.

    ReplyDelete

  16. Thanks for sharing this article this content is very significant for me I really appreciate you.

    ReplyDelete

  17. Thanks for such a good articles it really helped me after reading your posts.

    ReplyDelete
  18. Thanks for sharing your article It is very useful us and amazing blog for the users who want to learn.

    ReplyDelete
  19. users who want to learn.

    I appreciate your quality stuff, that was really so useful and informative.

    ReplyDelete
  20. Come to join our website The website is open all the time, convenient, safe, fast and extremely accurate.

    ReplyDelete

  21. Very interesting, wish to see much more like this. Thanks for sharing your information!

    ReplyDelete