Monday, June 4, 2012

My Eclipse/PyDev/git workflow for Sage

I want to briefly describe how I code for Sage so that others can comment on how to improve the workflow or maybe learn something from it.

For Python (and actually for all programming languages, meanwhile), I've always been using Eclipse as an IDE. With PyDev you get pretty nice Python syntax highlighting, code outlines, and features such as code completion, although I have never used that a lot. PyDev has even added Cython support for .pyx files recently—yeah!

I have added the complete Sage tree (which is located in /Applications/sage/ on my Mac) as an Eclipse project. As recommended and described earlier, I created a new branch ("sandbox") using sage -clone lattices; consequently, I'm working on the files in sage/devel/sage-lattices.

Test-driven development makes much sense in general, and especially when working on Sage:
  • Due to the fact that you have to rebuild Sage every time you change something, working completely "interactive" isn't possible.
  • You don't want to type the code to "bootstrap" your tests (e.g. creating lattices) every single time you change something.
  • Eventually, lots of examples are demanded in the documentation anyway.
This is why I usually add some examples that I want to be working in Python docstrings, then code something, and then run the tests. Running the tests is done by
./sage -bt devel/sage-lattices/sage/lattices/lattice.py
which will rebuild the changed components of Sage (compiling Cython code if necessary) and then run the tests defined in lattice.py. Usually it says "All tests passed!" in the end; otherwise it gives an overview of the test cases (examples) that failed.

To see how the documentation that is generated from docstrings looks like, I sometimes rebuild it. For some strange reasons, the documentation generator (Sphinx) does not always recognize that I changed something in the source files. To avoid having to rebuild the complete reference manual (which takes quite a while) every time, I created a small documentation part that only includes the lattices chapter. This is easily done by adding a folder to doc/en and creating files conf.py, index.rst, and lattices.rst therein. The latter basically contains
Lattices
========

.. automodule:: sage.lattices.lattice
   :members:
   :undoc-members:
   :show-inheritance:
Then, the relevant documentation can be rebuilt using
sage --docbuild lattices html -S -a,-E
(-S passes arguments on to Sphinx, -a,-E erase and recreate everything.)

On a side note: When building the HTML documentation on my Mac, I got a bunch of warnings saying "Incompatible library version: libfontconfig.1.dylib requires version 13.0.0 or later, but libfreetype.6.dylib provides version 10.0.0". It seems that the system-wide version of dvipng that I installed through Fink referenced a library "libfreetype" that was loaded through Sage, but with an insufficiently recent version. I simply resolved this by replacing the libfreetype* files in sage/local/lib with the Fink files from /sw/lib. Now the HTML documentation displays beautiful PNG graphics for LaTeX formulas.

I tried to use the integrated Eclipse/PyDev Python debugger with Sage, but ran into some problems. First, you have to launch Eclipse through the Sage shell, i.e. using
sage -sh
/path/to/eclipse
But then, after selecting sage-python as interpreter, when I tried to run the doctests in debug mode, I still got some weird ImportErrors. I could reproduce them using sage-python alone (leaving Eclipse out) and reported the issue on sage-trac.

Despite test-driven development, sometimes I still want to play around with things in Sage interactively.  To reload my lattice module after having done
from sage.lattices import lattice
at one point, I still have to rebuild everything (using the mentioned sage -bt ... command which tests things as well), and then reload it using
lattice = reload(lattice)
Then I can go on creating lattices, e.g. by
L = lattice.Lattice([[1, 0], [0, 1]])
and experiment with them.

When I am done working on something, I do a git commit to my sage repository using
git commit -am "my commit message"
Using the Eclipse git interface is terribly slow when working on the entire Sage tree (even though only a few files are checked in), so I'm using the terminal for this. Usually I also push commits to the github repository using
git push
immediately.

Usually I have at least three terminal windows open: one entirely for running the doctests, one for handling git, and one for "experiments" in Sage. To finally include a picture on this blog as well, here's how this might look like:

Please let me know if you have any suggestions.

Finally, some useful documentation links:

42 comments:

  1. Hey this is great! This is definitely helpful

    I've been working with sage + pydev on a little project for myself.

    This will definitely help me work on documentation!

    Thanks. Good look with the Lattice Coding!

    ReplyDelete
  2. Thanks for sharing, Reach your Goal & finally get a successful future. If you want to learn a high level language…………… Devops Training in Bangalore

    ReplyDelete
  3. It is really a great work and the way in which u r sharing the knowledge is excellent.
    Thanks for helping me to understand basic concepts. As a beginner your post help me a lot.Thanks for your informative article.
    Java Training in chennai |AWS Training in Chennai

    ReplyDelete
  4. Nice Blog on Eclipse/PyDev/git workflow for Sage..
    Its Really a Interesting blog i red..
    Devops Training

    ReplyDelete
  5. The knowledge of technology you have been sharing thorough this post is very much helpful to develop new idea. here by i also want to share this.
    Digital Marketing online training

    full stack developer training in pune

    full stack developer training in annanagar

    ReplyDelete
  6. It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command
    python training institute in chennai
    python training in Bangalore
    python training in pune
    python training institute in chennai
    python training in velachery
    python online training

    ReplyDelete
  7. I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post.is article.
    Blueprism training in Pune

    Blueprism online training

    Blue Prism Training in Pune

    ReplyDelete
  8. Good Post, I am a big believer in posting comments on sites to let the blog writers know that they ve added something advantageous to the world wide web.
    Data Science training in kalyan nagar | Data Science training in OMR
    selenium training in chennai | Data Science training in chennai
    Data science training in velachery | Data science online training

    ReplyDelete
  9. This is such a great post, and was thinking much the same myself. Another great update.
    java training in chennai

    java training in marathahalli | java training in btm layout

    ReplyDelete
  10. Thank you for benefiting from time to focus on this kind of, I feel firmly about it and also really like comprehending far more with this particular subject matter. In case doable, when you get know-how, is it possible to thoughts modernizing your site together with far more details? It’s extremely useful to me.

    Microsoft Azure online training
    Selenium online training
    Java online training
    Java Script online training
    Share Point online training

    ReplyDelete
  11. Attend The Python training in bangalore From ExcelR. Practical Python training in bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Python training in bangalore.
    python training in bangalore

    ReplyDelete
  12. Your info is really amazing with impressive content..Excellent blog with informative concept. Really I feel happy to see this useful blog, Thanks for sharing such a nice blog..
    If you are looking for any Data science Related information please visit our website data science institutes in bangalore page!

    ReplyDelete
  13. It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.
    cloud computing training in chennai | Cloud computing training class in chennai

    ReplyDelete
  14. Hi,
    Best article, very useful and well explanation. Your post is extremely incredible.Good job & thank you very much for the new information, i learned something new. Very well written. It was sooo good to read and usefull to improve knowledge. Who want to learn this information most helpful. One who wanted to learn this technology IT employees will always suggest you take Hadoop Admin Training Institute.

    ReplyDelete
  15. I’ve been searching for some decent stuff on the subject and haven't had any luck up until this point, You just got a new biggest fan!.. Keyword

    ReplyDelete
  16. I see the greatest contents on your blog and I extremely love reading them.
    full stack web development course

    ReplyDelete
  17. This post is so interactive and informative.keep update more information…
    German Classes in Anna Nagar
    German Classes in chennai



    ReplyDelete
  18. Всякий рассчитывает узнать собственную судьбу и видит определенные методы ворожбы гораздо больше достоверными. Гадание на любовь на любимую - способ понять приближающиеся явления постоянно манил род людской. Хиромантия позволяет просмотреть, что вас ждет в ближайшее время.

    ReplyDelete
  19. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one.
    data analytics course in hyderabad

    ReplyDelete