Running multiple versions of nodejs and npm on windows

I use NVM to run different versions on node on my Ubuntu development machine. So, when i wanted to do the same on my windows development machine, i was out of luck since NVM does not support windows. But nodejs is available for windows as just a downloadable .EXE file. This means that i can just include the required version of the tiny binary file in my projects and i should be good to go. Well, the ride got a little bumpy along way and am hoping that this post will help someone trying to do the same.

So, now that i had my node.exe in my project, the very first thing i wanted to do was to use NPM to install dependencies. How do I go about doing that? Well, i could download NPM as ZIP files from it’s releases on github. So, where do we put it? I had to created a “node_modules\npm” folder alongside the node.exe and extract the contents of the “npm-x.x.x” folder inside the npm zip file download into this folder. That’s not all, copy the files named “npm” and “npm.cmd” from the “node_modules\npm\bin” directory to the same directory as node.exe. Now, you should be able to use both nodejs and npm from the command line while referring to their qualified paths. Remember node and npm have installed in that project only and not globally. Invoke “npm install” and the dependencies in your packages.json should be installed into your local “node_modules”. Sounds like we have achieved our goal.

Great, we got everything working locally and checked in. The build server should be able to check it out and call the local npm and install all the project dependencies locally. Well, for me, things blew up on the build server. Why? NPM was having trouble with installing the node-sass package. This turned out to be a native dependency. The reason why everything worked on my machine but ended up failing on the build server was because I had VisualStudio 2015 installed on my development machine and it wasn’t installed on the build server (neither should it be IMHO). That is, my development machine had a C++ compiler available and the build server did not. This took me back to the days when I used to have the same exact problems with installing PyPI packages on windows machines. But, the Python community had already solved this problem using pre-built binaries in wheels distributions. BTW, if you are working in GIS & Python on windows,  Christoph Gohlke has a treasure trove for you here. Most of the consensus in online discussions seemed to be to install VisualStudio 2015 on the build server. But there is a glimmer of hope, Microsoft has recognized the issue and has made the Visual C++ Build Tools 2015 available exactly for this purpose (standalone c++ tools for build environments). You will also need to install Python 2.7 on the build server. The build server can run some scripts to configure the environment before it builds these native dependencies. First off, it needs to add Python to its PATH and configure NPM to use Python 2.7 with the following command

npm config set python python2.7

The Microsoft C++ build tools version can be set globally with this command

npm config set msvs_version 2015 –global

But, we can also set the C++ build tools version when like NPM install like shown here

npm install –msvs_version=2015

After all that, things should get working. I am hoping that this gets easier in the future if NPM moves to a Python wheels like pre-built binary ditribution for native dependencies.

Raster statistics + SIMD === winning

Recently, the .NET framework 4.6 enabled features that would allow us to leverage SIMD (Single Instruction Multiple Data) capabilities that have been present in our CPUs since the 2000s and deliver faster performance for calculating raster statistics.

We were using the .NET bindings for GDAL to calculate a few simple raster statistics for reporting needs. I got curious as to the kind of performance boost that we could be seeing if we leveraged SIMD for this purpose. So, I created a little console app to explore the performance gains. Here is what i observed.

To start off, we need to check if our CPU supports SIMD instructions. Although, I expect that everyone’s computer will support SIMD, here is simple way to check. Download and run CPU-Z. Keep an eye out for the SSE instructions as shown below.

Clone the git repo or just download the executable from here. And run the SIMD.exe file with a GeoTIFF file as a parameter. Needless to say, the console app is pretty rudimentary and was written solely to observe SIMD performance gains. I would highly encourage you to clone the repo and explore further if you are interested. The app currently just reads all the raster image into memory and sums all the values to produce a total sum. Not very useful, but i am hoping it can get you started in the right direction to explore further. The amount of code required or in other words to modify your current app to leverage should be manageable and shouldn’t require a complete rewrite. Also, currently the SIMD support in the .NET framework only works under 64-bit and in release mode. That had me stumped for a bit. But, there is very useful “IsHardwareAccelerated” Flag that helps you find out if your vectorized code is actually reaping the benefits of the SIMD hardware.

Raster X size: 28555
Raster Y size: 17939
Raster cell size: 512248145
Total is: 146097271
Time taken: 4568
SIMD accelerated: True
Another Total is: 146097271
Time taken: 1431

Here is the result from a run on a sample raster image. With the SIMD vectors, the execution went from 4568 to 1431 milliseconds. That’s almost 1/3 of the original time. Pretty tempting isn’t it?

The performance measure doesn’t really prove anything and is purely meant to satisfy my own curiosity. Your mileage may vary. To eek out any gains out of using SIMD vectors, you will need to be processing a large enough array that will justify the overhead of creating the SIMD vectors.

I realize that pretty much everything needs to be available in JavaScript to be relevant these days 🙂 So, if JavaScript is your cup of tea FireFox has experimental support for SIMD vectors. Soon we will be running raster statistics on the browser. Actually, we currently are running some raster statistics but without the SIMD bits. Hopefully, i will get around to writing about that soon.

Clone the repo here