Thanks! I appreciate the comment and the notes. I’ll be sure to upgrade Sage when ready and try out your suggestions.

We’re planning on upgrading scipy, numpy, and matplotlib to the latest releases in Sage 3.1 which should be out next week.

Regarding the speed of numpy.where(z<0) in Sage, this is because numpy and Sage data types don’t always play nicely together yet.

Sage has a light preparser that does things like replace things that would be Python integers with Sage’s Integer class (which is based on GMP). For example,

sage: preparse(“numpy.where(z<0)”)
‘numpy.where(z<Integer(0))’

I’m not sure where all the time is being spent in the numpy routine. But, there are a couple possible workarounds:

1) Turn off the Sage preprocessor with

sage: preparser(False)

2) Use the ‘r’ to tell the preparser not to change the Python integer to a Sage one:

sage: preparse(“”numpy.where(z<0r)”)
‘numpy.where(z<0)’

3) Replace Integer with int

sage: Integer = int.

The same sort of thing applies with Sage’s RealNumber class and Python’s float.

–Mike

again, thanks for stopping by and your insights. I tried your random number example on sage and straight Numpy and got some interesting results. First off timeit would not work in the sage environment, but a simple

z = standard_normal( (124,50,151) )
ind = numpy.where(z<0)

takes 20 seconds in Sage 3.0.5 (running Numpy 1.0.4)

while in standard python (running Numpy 1.1.0) that command is instantaneous.

In [31]: timeit z<0
100 loops, best of 3: 3.93 ms per loop

You are also dead on in regards to not using the major features of Sage which would differentiate it from just using Python+Packages. I feel the same way, but wanted to see if Sage would offer a “one stop shopping” where I could load data, run an EOF (matlab/python), run a Generalized Additive model (R), and plot a nice map (GMT) without having to push the data to three places.

At this point though, the use of an older version of Numpy really seems to be a problem for the applications that I want to run as I cannot use Basemap in matplotlib and the speed issues on indexing arrays.

So I do have Numpy/SciPy/IPython/Basemap setup (no RPy yet) and I think that I will just be content to use these and report that Sage just isn’t for me right now due to the Basemap and speed issues.

I am excited about the possibilities here, and I think that IPython is an amazing IDE to work in (especially ipython -pylab!). I actually *cheated* and used Matlab the other day, and it felt like a step backwards, especially when the Java-based IDE kept freezing on me (coupled with the fact that I *still* can’t run 2008a or 2008b due to my eth0 being diasbled).

Again, thanks for the comments and I’ll try to get some more playtime in with python today.

It’s strange that it took so long to index into your matrix . . . that size shouldn’t be too large at all.

In IPython on my 5-yr-old machine it’s near-instantaneous:

import numpy
z = numpy.random.standard_normal( (124,50,151) )

timeit z<0
# 100 loops, best of 3: 11.4 ms per loop

I wonder if the bottleneck is in loading the NetCDF data . . . maybe it’s doing some funky “lazy loading” and not actually retrieving the data until you do the data<0 test?

Sidenote: it was easier for me to make the transition easier from Matlab syntax when I found out vectorizing tricks like

data[data<0] = NaN

work with NumPy arrays.

So far it looks like everything you’re doing can be done without needing Sage. I haven’t installed it to test it, but poking around it seems the advantages of Sage are

1) its integration with computer algebra systems (http://www.sagemath.org/doc/html/tut/node46.html)

2) Notebooks

If you can get away with not using either of these, then it might be worth a shot to install the Python packages you’d use individually (NumPy, SciPy, matplotlib, basemap). Then you can always stay on top of the latest versions and bugfixes, and take advantage of high-quality documentation for the separate packages, for example matplotlib’s new docs at http://matplotlib.sourceforge.net/doc/html/users/index.html

-ryan