Applying Deep Learning at Cloud Scale, with Microsoft R Server & Azure Data Lake

Applying Deep Learning at Cloud Scale, w/ Microsoft R Server & Azure Data Lake

  • Figure 4: Generating training data in parallel using Microsoft R Server.
  • We present the final tagged test image in Figure 8 where cars and boats are labeled with red and green bounding boxes respectively; you can also download the image .
  • Each worker node returns a labelled list of moving window tile coordinates, which is then used to label the final test image in MRS running on HDInsight Spark edge node.
  • We compress 2.3 million training images from 8.9GB of raw PNG images to 5.1GB with im2rec binary in 10 minutes for optimal training performance.
  • MXNet DNN model training using NVIDIA Tesla K80 GPU using Microsoft R Server (MRS).

This post is by Max Kaznady, Data Scientist, Miguel Fierro, Data Scientist, Richin Jain, Solution Architect, T. J. Hazen, Principal Data Scientist Manager, and Tao Wu, Principal Data Scientist Manager, all at Microsoft.

Continue reading “Applying Deep Learning at Cloud Scale, with Microsoft R Server & Azure Data Lake”