Monday, June 4, 2012

My Eclipse/PyDev/git workflow for Sage

I want to briefly describe how I code for Sage so that others can comment on how to improve the workflow or maybe learn something from it.

For Python (and actually for all programming languages, meanwhile), I've always been using Eclipse as an IDE. With PyDev you get pretty nice Python syntax highlighting, code outlines, and features such as code completion, although I have never used that a lot. PyDev has even added Cython support for .pyx files recently—yeah!

I have added the complete Sage tree (which is located in /Applications/sage/ on my Mac) as an Eclipse project. As recommended and described earlier, I created a new branch ("sandbox") using sage -clone lattices; consequently, I'm working on the files in sage/devel/sage-lattices.

Test-driven development makes much sense in general, and especially when working on Sage:
  • Due to the fact that you have to rebuild Sage every time you change something, working completely "interactive" isn't possible.
  • You don't want to type the code to "bootstrap" your tests (e.g. creating lattices) every single time you change something.
  • Eventually, lots of examples are demanded in the documentation anyway.
This is why I usually add some examples that I want to be working in Python docstrings, then code something, and then run the tests. Running the tests is done by
./sage -bt devel/sage-lattices/sage/lattices/lattice.py
which will rebuild the changed components of Sage (compiling Cython code if necessary) and then run the tests defined in lattice.py. Usually it says "All tests passed!" in the end; otherwise it gives an overview of the test cases (examples) that failed.

To see how the documentation that is generated from docstrings looks like, I sometimes rebuild it. For some strange reasons, the documentation generator (Sphinx) does not always recognize that I changed something in the source files. To avoid having to rebuild the complete reference manual (which takes quite a while) every time, I created a small documentation part that only includes the lattices chapter. This is easily done by adding a folder to doc/en and creating files conf.py, index.rst, and lattices.rst therein. The latter basically contains
Lattices
========

.. automodule:: sage.lattices.lattice
   :members:
   :undoc-members:
   :show-inheritance:
Then, the relevant documentation can be rebuilt using
sage --docbuild lattices html -S -a,-E
(-S passes arguments on to Sphinx, -a,-E erase and recreate everything.)

On a side note: When building the HTML documentation on my Mac, I got a bunch of warnings saying "Incompatible library version: libfontconfig.1.dylib requires version 13.0.0 or later, but libfreetype.6.dylib provides version 10.0.0". It seems that the system-wide version of dvipng that I installed through Fink referenced a library "libfreetype" that was loaded through Sage, but with an insufficiently recent version. I simply resolved this by replacing the libfreetype* files in sage/local/lib with the Fink files from /sw/lib. Now the HTML documentation displays beautiful PNG graphics for LaTeX formulas.

I tried to use the integrated Eclipse/PyDev Python debugger with Sage, but ran into some problems. First, you have to launch Eclipse through the Sage shell, i.e. using
sage -sh
/path/to/eclipse
But then, after selecting sage-python as interpreter, when I tried to run the doctests in debug mode, I still got some weird ImportErrors. I could reproduce them using sage-python alone (leaving Eclipse out) and reported the issue on sage-trac.

Despite test-driven development, sometimes I still want to play around with things in Sage interactively.  To reload my lattice module after having done
from sage.lattices import lattice
at one point, I still have to rebuild everything (using the mentioned sage -bt ... command which tests things as well), and then reload it using
lattice = reload(lattice)
Then I can go on creating lattices, e.g. by
L = lattice.Lattice([[1, 0], [0, 1]])
and experiment with them.

When I am done working on something, I do a git commit to my sage repository using
git commit -am "my commit message"
Using the Eclipse git interface is terribly slow when working on the entire Sage tree (even though only a few files are checked in), so I'm using the terminal for this. Usually I also push commits to the github repository using
git push
immediately.

Usually I have at least three terminal windows open: one entirely for running the doctests, one for handling git, and one for "experiments" in Sage. To finally include a picture on this blog as well, here's how this might look like:

Please let me know if you have any suggestions.

Finally, some useful documentation links:

90 comments:

  1. Hey this is great! This is definitely helpful

    I've been working with sage + pydev on a little project for myself.

    This will definitely help me work on documentation!

    Thanks. Good look with the Lattice Coding!

    ReplyDelete
  2. Thanks for sharing, Reach your Goal & finally get a successful future. If you want to learn a high level language…………… Devops Training in Bangalore

    ReplyDelete
  3. It is really a great work and the way in which u r sharing the knowledge is excellent.
    Thanks for helping me to understand basic concepts. As a beginner your post help me a lot.Thanks for your informative article.
    Java Training in chennai |AWS Training in Chennai

    ReplyDelete
  4. Nice Blog on Eclipse/PyDev/git workflow for Sage..
    Its Really a Interesting blog i red..
    Devops Training

    ReplyDelete
  5. The knowledge of technology you have been sharing thorough this post is very much helpful to develop new idea. here by i also want to share this.
    Digital Marketing online training

    full stack developer training in pune

    full stack developer training in annanagar

    ReplyDelete
  6. It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command
    python training institute in chennai
    python training in Bangalore
    python training in pune
    python training institute in chennai
    python training in velachery
    python online training

    ReplyDelete
  7. I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post.is article.
    Blueprism training in Pune

    Blueprism online training

    Blue Prism Training in Pune

    ReplyDelete
  8. Good Post, I am a big believer in posting comments on sites to let the blog writers know that they ve added something advantageous to the world wide web.
    Data Science training in kalyan nagar | Data Science training in OMR
    selenium training in chennai | Data Science training in chennai
    Data science training in velachery | Data science online training

    ReplyDelete
  9. This is such a great post, and was thinking much the same myself. Another great update.
    java training in chennai

    java training in marathahalli | java training in btm layout

    ReplyDelete
  10. Greetings. I know this is somewhat off-topic, but I was wondering if you knew where I could get a captcha plugin for my comment form? I’m using the same blog platform like yours, and I’m having difficulty finding one? Thanks a lot.

    AWS Interview Questions And Answers

    AWS Tutorial |Learn Amazon Web Services Tutorials |AWS Tutorial For Beginners


    AWS Online Training | Online AWS Certification Course - Gangboard

    AWS Training in Toronto| Amazon Web Services Training in Toronto, Canada

    ReplyDelete
  11. Thank you for benefiting from time to focus on this kind of, I feel firmly about it and also really like comprehending far more with this particular subject matter. In case doable, when you get know-how, is it possible to thoughts modernizing your site together with far more details? It’s extremely useful to me.

    Microsoft Azure online training
    Selenium online training
    Java online training
    Java Script online training
    Share Point online training

    ReplyDelete
  12. Thanks For Sharing The Information The Information Shared Is Very Valuable Please Keep Updating Us Time Just Went On Reading The article Python Online Course Hadoop Online Course Aws Online Course Data Science Online Course

    ReplyDelete
  13. Attend The Python training in bangalore From ExcelR. Practical Python training in bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Python training in bangalore.
    python training in bangalore

    ReplyDelete
  14. I am looking for and I love to post a comment Python training in punethat "The content of your post is awesome" Great work!

    ReplyDelete
  15. Your info is really amazing with impressive content..Excellent blog with informative concept. Really I feel happy to see this useful blog, Thanks for sharing such a nice blog..
    If you are looking for any Data science Related information please visit our website data science institutes in bangalore page!

    ReplyDelete
  16. I learned World's Trending Technology from certified experts for free of cost. I Got a job in decent Top MNC Company with handsome 14 LPA salary, I have learned the World's Trending Technology from best training institute for data science in bangalore experts who know advanced concepts which can help to solve any type of Real-time issues in the field of Python. Really worth trying Instant Approval Blog Commenting Sites

    ReplyDelete
  17. Attend The Artificial Intelligence course From ExcelR. Practical Artificial Intelligence course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Artificial Intelligence course.
    ExcelR Artificial Intelligence course

    ReplyDelete
  18. It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.
    cloud computing training in chennai | Cloud computing training class in chennai

    ReplyDelete
  19. Hi,
    Best article, very useful and well explanation. Your post is extremely incredible.Good job & thank you very much for the new information, i learned something new. Very well written. It was sooo good to read and usefull to improve knowledge. Who want to learn this information most helpful. One who wanted to learn this technology IT employees will always suggest you take Hadoop Admin Training Institute.

    ReplyDelete
  20. I’ve been searching for some decent stuff on the subject and haven't had any luck up until this point, You just got a new biggest fan!.. Keyword

    ReplyDelete
  21. I was searching for exactly the same information.Thanks for sharing.Good work.Keep it up.These days Big data is trending technology.“Without big data, companies are blind and deaf, wandering out onto the web like deer on a freeway.”If you are looking for any online courses on big data visit our site.
    Big Data Hadoop Online Training Courses

    ReplyDelete
  22. The concept has been explained very well. Gained clear knowledge about the prominence of AWS
    AWS Training in Hyderabad

    ReplyDelete
  23. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here

    Big Data Analytics Training In Hyderabad
    Big Data Analytics Course In Hyderabad

    ReplyDelete
  24. Thank you for taking the time and sharing this information with us. It was indeed very helpful and insightful while being straight forward and to the point.
    Python Training In Hyderabad

    ReplyDelete
  25. wonderful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article resolved my all queries.
    Data Science Course

    ReplyDelete
  26. It's a wonderful post and very helpful, thanks for all this information about Java. You are including better information regarding this topic in an effective way.Thank you so much.
    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

    ReplyDelete
  27. Great blog. it’s really helpful. Thank you so much for this incredible guide, . Keep sharing on updated blog posts.
    Machine Learning Training in Hyderabad

    ReplyDelete
  28. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
    Correlation vs Covariance
    Simple linear regression

    ReplyDelete
  29. Attend The Artificial Intelligence course From ExcelR. Practical Artificial Intelligence course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Artificial Intelligence course.
    Artificial Intelligence Course

    ReplyDelete
  30. Thank you so much for this incredible guide. This has given me so much information
    AI Training in Hyderabad

    ReplyDelete
  31. Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.You can also check my articles as well.

    Data Science In Banglore With Placements
    Data Science Course In Bangalore
    Data Science Training In Bangalore
    Best Data Science Courses In Bangalore
    Data Science Institute In Bangalore

    Thank you..

    ReplyDelete
  32. Really Very Infromative Post , Thanks For Sharing The Information With Us.
    Best AWS Training Institute in Hyderabad

    ReplyDelete
  33. The development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. machine learning projects for final year In case you will succeed, you have to begin building machine learning projects in the near future.

    Projects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.


    Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.


    The Nodejs Training Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

    ReplyDelete

  34. Very good points you wrote here..Great stuff...I think you've made some truly interesting points.Keep up the good work.
    360digitmg

    ReplyDelete
  35. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website

    DevOps Training in Hyderabad

    ReplyDelete
  36. Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
    Simple Linear Regression
    Correlation vs covariance
    data science interview questions
    KNN Algorithm
    Logistic Regression explained

    ReplyDelete
  37. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
    tally training in chennai

    hadoop training in chennai

    sap training in chennai

    oracle training in chennai

    angular js training in chennai

    ReplyDelete
  38. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
    salesforce training in chennai

    software testing training course in chennai

    robotic process automation rpa training in chennai

    blockchain training in chennai

    devops training in chennai

    ReplyDelete
  39. Truly mind blowing blog went amazed with the subject they have developed the content. These kind of posts really helpful to gain the knowledge of unknown things which surely triggers to motivate and learn the new innovative contents. Hope you deliver the similar successive contents forthcoming as well.

    data science in bangalore

    ReplyDelete