Best practices of orchestrating Python and R code in ML projects

Best practices of orchestrating #Python and #rstats code in #MachineLearning projects

  • Instead of arguing about Python vs R I will examine the best practices of integrating both languages in one data science project.
  • Today, data scientists are generally divided among two languages — some prefer R, some prefer Python.
  • Usually algorithms used for classification or regression are implemented in both languages and some scientist are using R while some of them preferring Python.
  • Instead of using logistic regression in R we will write Python jobs in which we will try to use random forest as training model.
  • py is presented below: – – Also here we are adding code for download necessary R and Python codes from above (clone the Githubrepository): – – Our dependency graph of this data science project look like this: – – Now lets see how it is possible to speed up and simplify…


Instead of arguing about Python vs R I will examine the best practices of integrating both languages in one data science project.

Today, data scientists are generally divided among two languages — some prefer R, some prefer Python. I will not try to explain in this article which one is better. Instead of that I will try to find an answer to a question: “What is the best way to integrate both languages in one data science project? What are the best practices?”. Beside git and shell scripting additional tools are developed to facilitate the development of predictive model in a multi-language environments. For fast data exchange between R and Python let’s use binary data file format Feather. Another language agnostic tool DVC can make the research reproducible — let’s use DVC to orchestrate R and Python code instead of a regular shell scripts.

Both R and Python are having powerful libraries/packages used for predictive modeling. Usually algorithms used for classification or regression are implemented in both languages and some scientist are using R while some of them preferring Python. In an example that was explained in previous tutorialtarget variable was binary output and logistic regression was used as a training algorithm. One of the algorithms that could also be used for prediction is a popular Random Forest algorithm which is implemented in both programming languages. Because of performances it was decided that Random Forest classifier should be implemented in Python (it shows better performances than random forest package in R).

We will use the same example from previous blog story, add some Python codes and explain how Feather and DVC can simplify the development process in this combined…

Best practices of orchestrating Python and R code in ML projects