<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Parallel | Julián Cabezas Peña</title><link>https://www.juliancabezas.com/en/tags/parallel/</link><atom:link href="https://www.juliancabezas.com/en/tags/parallel/index.xml" rel="self" type="application/rss+xml"/><description>Parallel</description><generator>HugoBlox Kit (https://hugoblox.com)</generator><language>en-US</language><lastBuildDate>Thu, 24 Jan 2019 00:00:00 +0000</lastBuildDate><image><url>https://www.juliancabezas.com/media/icon_hu_a22537ec31d66339.png</url><title>Parallel</title><link>https://www.juliancabezas.com/en/tags/parallel/</link></image><item><title>A function for parallel processing on Windows and servers (R parallelisation, part 2)</title><link>https://www.juliancabezas.com/en/post/parallel-cluster-function/</link><pubDate>Thu, 24 Jan 2019 00:00:00 +0000</pubDate><guid>https://www.juliancabezas.com/en/post/parallel-cluster-function/</guid><description>&lt;div id="TOC"&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#el-problema-crear-clusters-es-tedioso"&gt;The problem: creating clusters is tedious&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#la-solucion-una-funcion-para-clear-clusters"&gt;The solution: a function to create clusters&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#un-pequeno-y-reciclado-ejemplo"&gt;A small, recycled example&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;
&lt;p&gt;This short tutorial is the second part of my introduction to parallelising R on Windows. &lt;a href="https://www.juliancabezas.com/en/post/parallel-r-windows/"&gt;I recommend reading the first part&lt;/a&gt;.&lt;/p&gt;
&lt;div id="el-problema-crear-clusters-es-tedioso" class="section level2"&gt;
&lt;h2&gt;The problem: creating clusters is tedious&lt;/h2&gt;
&lt;p&gt;Creating clusters can be tedious: loading packages one by one and exporting datasets is slow, and it is easy to forget a crucial package. We want a more convenient approach. We may also be working on a server without Internet access, which complicates package installation.&lt;/p&gt;
&lt;/div&gt;
&lt;div id="la-solucion-una-funcion-para-clear-clusters" class="section level2"&gt;
&lt;h2&gt;The solution: a function to create clusters&lt;/h2&gt;
&lt;p&gt;I created a function, available in &lt;a href="https://github.com/juliancabezas/random_functions/blob/master/create_cluster"&gt;this GitHub file&lt;/a&gt;. To give credit where it is due, it is inspired by &lt;a href="https://github.com/nathanvan/parallelsugar"&gt;parallelsugar&lt;/a&gt;, with a few modifications.&lt;/p&gt;
&lt;p&gt;The function creates a cluster with &lt;em&gt;n&lt;/em&gt; nodes, specified by the &lt;em&gt;nnodes&lt;/em&gt; argument.&lt;/p&gt;
&lt;p&gt;create_cluster is designed to work on servers. When they have no Internet connection, packages can be installed in a shared folder and loaded from there. Use &lt;em&gt;libroute&lt;/em&gt; to specify the library path.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;create_cluster &amp;lt;- function(nnodes, libroute = NULL) {
&lt;h1 id="nnodes-number-of-nodes"&gt;nnodes: Number of nodes&lt;/h1&gt;
&lt;h1 id="libroute-provided-library-route-different-of-the-default-library"&gt;libroute: Provided library route (different of the default library)&lt;/h1&gt;
&lt;p&gt;tryCatch({
cl &amp;lt;- parallel::makeCluster(nnodes) # Create an empty cluster with n nodes&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;loaded.package.names &amp;amp;lt;- c( # Find the loaded packages in the session
sessionInfo()$basePkgs, # base packages
names(sessionInfo()$otherPkgs)
) # aditional loaded packages
# Send the packages and the complete environment to the cluster
parallel::clusterExport(cl,
&amp;amp;quot;loaded.package.names&amp;amp;quot;, # loaded packages
envir = environment()
) # &amp;amp;quot;Environment&amp;amp;quot;
## Charge the packages in all nodes
if (is.null(libroute)) { # Use default library location
parallel::parLapply(cl, 1:length(cl), function(xx) {
lapply(loaded.package.names, function(yy) {
library(yy, character.only = TRUE)
})
})
} else {
# If we provide a library location use it (useful when working in remote machines)
parallel::parLapply(cl, 1:length(cl), function(xx) {
lapply(loaded.package.names, function(yy) {
library(yy, character.only = TRUE, lib.loc = libroute)
})
})
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;})
return(cl) # The result is the cluster
}&lt;/code&gt;&lt;/pre&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id="un-pequeno-y-reciclado-ejemplo" class="section level2"&gt;
&lt;h2&gt;A small, recycled example&lt;/h2&gt;
&lt;p&gt;In the example from the previous tutorial, we used &lt;em&gt;clusterCall&lt;/em&gt; to load &lt;em&gt;randomForest&lt;/em&gt;. Here, if the package and any other dependencies are already loaded in our session, we can export them to the nodes in one step.&lt;/p&gt;
&lt;p&gt;First, load the package and create the function:&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;library(randomForest)
&lt;p&gt;rf_boot_fun &amp;lt;- function(iter, data, n_sample) {
ind &amp;lt;- sample(1:nrow(data), size = n_sample)
data_sample &amp;lt;- data[ind, ]&lt;/p&gt;
&lt;h1 id="fit-the-random-forest-model"&gt;Fit the Random Forest model&lt;/h1&gt;
&lt;p&gt;model1 &amp;lt;- randomForest(Sepal.Length ~ Petal.Length, data = data_sample)&lt;/p&gt;
&lt;h1 id="extract-the-pseudo-r2"&gt;Extract the pseudo-R2&lt;/h1&gt;
&lt;p&gt;R2 &amp;lt;- model1$rsq[length(model1$rsq)]&lt;/p&gt;
&lt;h1 id="extract-the-mse"&gt;Extract the MSE&lt;/h1&gt;
&lt;p&gt;MSE &amp;lt;- model1$mse[length(model1$mse)]
results &amp;lt;- c(&amp;quot;R2&amp;quot; = R2, &amp;quot;MSE&amp;quot; = MSE)
return(results)
}&lt;/code&gt;&lt;/pre&gt;&lt;/p&gt;
&lt;p&gt;Now I can apply the function with &lt;em&gt;parLapply&lt;/em&gt;.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;library(parallel)
&lt;h1 id="define-a-library-path-optional"&gt;Define a library path (optional)&lt;/h1&gt;
&lt;p&gt;ruta_libreria &amp;lt;- &amp;quot;C:/Users/Julian/Documents/R/win-library/3.5&amp;quot;&lt;/p&gt;
&lt;h1 id="this-can-simply-be-nuevoclus-create_cluster4"&gt;This can simply be nuevoclus&amp;lt;-create_cluster(4)&lt;/h1&gt;
&lt;p&gt;nuevoclus &amp;lt;- create_cluster(4, ruta_libreria)&lt;/p&gt;
&lt;h1 id="10000-iterations-of-our-function-using-the-iris-dataset"&gt;10,000 iterations of our function using the &amp;quot;iris&amp;quot; dataset&lt;/h1&gt;
&lt;p&gt;results &amp;lt;- parLapply(nuevoclus, 1:10000, rf_boot_fun, data = iris, n_sample = 100)&lt;/p&gt;
&lt;p&gt;stopCluster(nuevoclus) # Close the cluster; do not forget this step&lt;/p&gt;
&lt;h1 id="the-first-two-results"&gt;The first two results&lt;/h1&gt;
&lt;p&gt;head(results, n = 2)&lt;/code&gt;&lt;/pre&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;## [[1]]
## R2 MSE
## 0.778244 0.174310
##
## [[2]]
## R2 MSE
## 0.6912495 0.1865890&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Done—parallelisation is much easier now!&lt;/p&gt;
&lt;div class="figure"&gt;
&lt;img src="https://media.giphy.com/media/uTuLngvL9p0Xe/giphy.gif" /&gt;
&lt;/div&gt;
&lt;p&gt;&lt;strong&gt;I hope you find this useful!&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;I welcome your comments and questions!&lt;/strong&gt;&lt;/p&gt;
&lt;/div&gt;</description></item><item><title>Parallelising R on Windows with the parallel package</title><link>https://www.juliancabezas.com/en/post/parallel-r-windows/</link><pubDate>Mon, 07 Jan 2019 00:00:00 +0000</pubDate><guid>https://www.juliancabezas.com/en/post/parallel-r-windows/</guid><description>&lt;div id="TOC"&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#por-que-paralelizar"&gt;Why parallelise?&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#convertir-tu-loop-en-una-funcion"&gt;Turn your loop into a function&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#usando-la-funcion-lapply"&gt;Using lapply&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#el-paquete-parallel"&gt;The parallel package&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#un-ejemplo-mas-realista-usando-machine-learning"&gt;A more realistic example: machine learning&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;
&lt;div id="por-que-paralelizar" class="section level2"&gt;
&lt;h2&gt;Why parallelise?&lt;/h2&gt;
&lt;p&gt;You have been asked to run code that takes 20 hours to finish. It applies the same function to different parameters or datasets many, &lt;strong&gt;many&lt;/strong&gt; times.&lt;/p&gt;
&lt;p&gt;That might not seem like a problem, but requests often arrive &lt;del&gt;yesterday&lt;/del&gt; without 20 hours of notice. If you are wondering why your PC or workstation has 4, 8 or 16 cores, this tutorial is for you!&lt;/p&gt;
&lt;p&gt;Parallelising R processes can be difficult. Add &lt;del&gt;bad luck&lt;/del&gt; Windows to the equation, and it can feel almost impossible.&lt;/p&gt;
&lt;p&gt;In this tutorial (&lt;strong&gt;the first on this website!&lt;/strong&gt;), we will introduce parallel processing in R on Windows. We will start with the &lt;strong&gt;apply&lt;/strong&gt; family, explore the parallel package and create a Windows cluster, before finishing with a common &lt;em&gt;machine learning&lt;/em&gt; application.&lt;/p&gt;
&lt;div id="nivel-de-este-tutorial" class="section level4"&gt;
&lt;h4&gt;Tutorial level&lt;/h4&gt;
&lt;p&gt;This tutorial assumes you are an intermediate or advanced R user.&lt;/p&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div id="convertir-tu-loop-en-una-funcion" class="section level2"&gt;
&lt;h2&gt;Turn your loop into a function&lt;/h2&gt;
&lt;p&gt;We will use the &lt;strong&gt;iris&lt;/strong&gt; dataset, familiar to many R users. It contains petal and sepal length and width measurements for three flower species.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;data(iris)
knitr::kable(head(iris))&lt;/code&gt;&lt;/pre&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr class="header"&gt;
&lt;th align="right"&gt;Sepal.Length&lt;/th&gt;
&lt;th align="right"&gt;Sepal.Width&lt;/th&gt;
&lt;th align="right"&gt;Petal.Length&lt;/th&gt;
&lt;th align="right"&gt;Petal.Width&lt;/th&gt;
&lt;th align="left"&gt;Species&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr class="odd"&gt;
&lt;td align="right"&gt;5.1&lt;/td&gt;
&lt;td align="right"&gt;3.5&lt;/td&gt;
&lt;td align="right"&gt;1.4&lt;/td&gt;
&lt;td align="right"&gt;0.2&lt;/td&gt;
&lt;td align="left"&gt;setosa&lt;/td&gt;
&lt;/tr&gt;
&lt;tr class="even"&gt;
&lt;td align="right"&gt;4.9&lt;/td&gt;
&lt;td align="right"&gt;3.0&lt;/td&gt;
&lt;td align="right"&gt;1.4&lt;/td&gt;
&lt;td align="right"&gt;0.2&lt;/td&gt;
&lt;td align="left"&gt;setosa&lt;/td&gt;
&lt;/tr&gt;
&lt;tr class="odd"&gt;
&lt;td align="right"&gt;4.7&lt;/td&gt;
&lt;td align="right"&gt;3.2&lt;/td&gt;
&lt;td align="right"&gt;1.3&lt;/td&gt;
&lt;td align="right"&gt;0.2&lt;/td&gt;
&lt;td align="left"&gt;setosa&lt;/td&gt;
&lt;/tr&gt;
&lt;tr class="even"&gt;
&lt;td align="right"&gt;4.6&lt;/td&gt;
&lt;td align="right"&gt;3.1&lt;/td&gt;
&lt;td align="right"&gt;1.5&lt;/td&gt;
&lt;td align="right"&gt;0.2&lt;/td&gt;
&lt;td align="left"&gt;setosa&lt;/td&gt;
&lt;/tr&gt;
&lt;tr class="odd"&gt;
&lt;td align="right"&gt;5.0&lt;/td&gt;
&lt;td align="right"&gt;3.6&lt;/td&gt;
&lt;td align="right"&gt;1.4&lt;/td&gt;
&lt;td align="right"&gt;0.2&lt;/td&gt;
&lt;td align="left"&gt;setosa&lt;/td&gt;
&lt;/tr&gt;
&lt;tr class="even"&gt;
&lt;td align="right"&gt;5.4&lt;/td&gt;
&lt;td align="right"&gt;3.9&lt;/td&gt;
&lt;td align="right"&gt;1.7&lt;/td&gt;
&lt;td align="right"&gt;0.4&lt;/td&gt;
&lt;td align="left"&gt;setosa&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Suppose a colleague wants to fit a linear model relating petals to sepals. Because the sample is small (150 individuals), you are asked to run 20,000 bootstrap iterations to examine the model coefficients and R&lt;sup&gt;2&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;We can do this with a loop that draws a sample of 100 individuals (two-thirds of the dataset), fits a linear model, and saves the R&lt;sup&gt;2&lt;/sup&gt; and coefficients in a data frame.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;# Create an empty data frame to store the results
results &amp;lt;- data.frame()
&lt;p&gt;system.time({
for (i in 1:20000) {
# Draw a sample of 100 numbers between 1 and 150
ind &amp;lt;- sample(1:nrow(iris), size = 100)
iris_sample &amp;lt;- iris[ind, ]
# Fit a linear model using petal and sepal measurements
model1 &amp;lt;- lm(iris_sample[, &amp;quot;Sepal.Length&amp;quot;] ~ iris_sample[, &amp;quot;Petal.Length&amp;quot;])&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;# Extract R2 and the coefficients
R2 &amp;amp;lt;- summary(model1)$r.squared
coef &amp;amp;lt;- coefficients(model1)
# Combine everything in the results data frame
results &amp;amp;lt;- rbind(results, c(R2, coef))
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;}
})&lt;/code&gt;&lt;/pre&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;## user system elapsed
## 20.90 0.03 20.95&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The algorithm takes just over 20 seconds. That may be acceptable now, but what if tomorrow you need to do the same thing with half a million observations?&lt;/p&gt;
&lt;p&gt;The problem is that the loop uses only one processor core, running one iteration after another. This is not very efficient.&lt;/p&gt;
&lt;p&gt;The first step towards parallelisation is to turn the loop into a function. Here the only argument will be an iteration vector, “iter”, which lets us request more iterations.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;lm_boot_fun &amp;lt;- function(iter) {
ind &amp;lt;- sample(1:nrow(iris), size = 100)
iris_sample &amp;lt;- iris[ind, ]
model1 &amp;lt;- lm(iris_sample[, &amp;quot;Sepal.Length&amp;quot;] ~ iris_sample[, &amp;quot;Petal.Length&amp;quot;])
R2 &amp;lt;- summary(model1)$r.squared
coef &amp;lt;- coefficients(model1)
# Create a vector of results
results &amp;lt;- c(&amp;quot;R2&amp;quot; = R2, coef)
# Return the function output
return(results)
}&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let us test the function. As the following code shows, it returns a result regardless of the argument we pass.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;result &amp;lt;- lm_boot_fun(1)
result&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## R2 (Intercept)
## 0.7554039 4.3353075
## iris_sample[, &amp;quot;Petal.Length&amp;quot;]
## 0.3999710&lt;/code&gt;&lt;/pre&gt;
&lt;pre class="r"&gt;&lt;code&gt;result &amp;lt;- lm_boot_fun(56876)
result&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## R2 (Intercept)
## 0.7393922 4.3426829
## iris_sample[, &amp;quot;Petal.Length&amp;quot;]
## 0.3939433&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Passing a vector still returns just one result, because “results” is overwritten on every iteration.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;result &amp;lt;- lm_boot_fun(1:10)
result&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## R2 (Intercept)
## 0.7732318 4.2822007
## iris_sample[, &amp;quot;Petal.Length&amp;quot;]
## 0.4256894&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id="usando-la-funcion-lapply" class="section level2"&gt;
&lt;h2&gt;Using lapply&lt;/h2&gt;
&lt;p&gt;To keep all the results, we can use lapply.&lt;/p&gt;
&lt;p&gt;lapply belongs to the apply family (apply, sapply, vapply, lapply and mapply), one of R’s most powerful tools—and one I wish I had learned earlier. This tutorial is not about the whole family, so we will focus on lapply.&lt;/p&gt;
&lt;p&gt;lapply applies another function, such as the one we created, to a list or vector of arguments, and returns a list of the same length.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;iter &amp;lt;- seq(1, 20000)
system.time({
results &amp;lt;- lapply(iter, lm_boot_fun)
})&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## user system elapsed
## 15.05 0.00 15.07&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Processing time has already improved a little, but we want to make it much faster. One detail to handle is that lapply returns a list.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;head(results, n = 3)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [[1]]
## R2 (Intercept)
## 0.7795166 4.2839537
## iris_sample[, &amp;quot;Petal.Length&amp;quot;]
## 0.4123361
##
## [[2]]
## R2 (Intercept)
## 0.7683068 4.2132627
## iris_sample[, &amp;quot;Petal.Length&amp;quot;]
## 0.4311986
##
## [[3]]
## R2 (Intercept)
## 0.7909086 4.2600338
## iris_sample[, &amp;quot;Petal.Length&amp;quot;]
## 0.4163103&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To convert this list into a data frame, use unlist to build a matrix and then convert it. Here we specify three columns.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;results &amp;lt;- data.frame(matrix(unlist(results), ncol = 3, byrow = T))
head(results, n = 2)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## X1 X2 X3
## 1 0.7795166 4.283954 0.4123361
## 2 0.7683068 4.213263 0.4311986&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can find more elegant solutions in &lt;a href="https://stackoverflow.com/questions/4227223/r-list-to-data-frame"&gt;this Stack Overflow discussion&lt;/a&gt;.&lt;/p&gt;
&lt;/div&gt;
&lt;div id="el-paquete-parallel" class="section level2"&gt;
&lt;h2&gt;The parallel package&lt;/h2&gt;
&lt;p&gt;The parallel package is included with R. It provides parallel versions of functions in the apply family. Before using them, we need to find out how many cores are available.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;library(parallel)
numCores &amp;lt;- detectCores()
numCores&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] 8&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We have eight logical cores! As a rule of thumb for this example, we will use half of them (four). On a Unix-like system, such as Linux or macOS, we could use mclapply. However, mclapply relies on &lt;em&gt;forking&lt;/em&gt;, which creates a copy of the process and is available only on Unix-like systems.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;iter &amp;lt;- seq(1, 20000)
system.time({
results &amp;lt;- mclapply(iter, lm_boot_fun, mc.cores = 7)
})&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Error in mclapply(iter, lm_boot_fun, mc.cores = 7): &amp;#39;mc.cores&amp;#39; &amp;gt; 1 is not supported on Windows&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Timing stopped at: 0 0 0&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;On Windows, we will create a cluster manually using makeCluster. The resulting object becomes an argument to parLapply. In this case, we will use four cores.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;cl &amp;lt;- makeCluster(4) # Create the cluster
system.time(
results &amp;lt;- parLapply(cl, iter, lm_boot_fun)
)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## user system elapsed
## 0.03 0.00 4.76&lt;/code&gt;&lt;/pre&gt;
&lt;pre class="r"&gt;&lt;code&gt;stopCluster(cl) # Close the cluster; do not forget this step&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;strong&gt;Wow!&lt;/strong&gt; We have substantially reduced the bootstrap processing time. We can enjoy a cup of mate—after shutting down the cluster with stopCluster.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Great!&lt;/strong&gt; &lt;img src="https://media.giphy.com/media/4xpB3eE00FfBm/giphy.gif" /&gt;&lt;/p&gt;
&lt;p&gt;We could stop here, but this is a very simple example. In practice, we often use datasets stored on our computers and five to ten packages.&lt;/p&gt;
&lt;div id="complejizando-la-funcion" class="section level3"&gt;
&lt;h3&gt;Making the function more complex&lt;/h3&gt;
&lt;p&gt;Let us pretend iris is a dataset stored on our computer. We will also add two arguments to the function: the dataset and the sample size.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;# Write the iris dataset to disk
write.csv(iris, file = &amp;quot;iris.csv&amp;quot;)
&lt;h1 id="read-the-dataset-created-above"&gt;Read the dataset created above&lt;/h1&gt;
&lt;p&gt;datos_pc &amp;lt;- read.csv(&amp;quot;iris.csv&amp;quot;)&lt;/p&gt;
&lt;p&gt;lm_boot_fun2 &amp;lt;- function(iter, data, n_sample) {&lt;/p&gt;
&lt;h1 id="data-and-n_sample-are-now-function-arguments"&gt;data and n_sample are now function arguments&lt;/h1&gt;
&lt;p&gt;ind &amp;lt;- sample(1:nrow(data), size = n_sample)
data_sample &amp;lt;- data[ind, ]
model1 &amp;lt;- lm(data_sample[, &amp;quot;Sepal.Length&amp;quot;] ~ data_sample[, &amp;quot;Petal.Length&amp;quot;])
R2 &amp;lt;- summary(model1)$r.squared
coef &amp;lt;- coefficients(model1)
results &amp;lt;- c(&amp;quot;R2&amp;quot; = R2, coef)
return(results)
}&lt;/code&gt;&lt;/pre&gt;&lt;/p&gt;
&lt;p&gt;Now we can repeat the operation using the additional arguments of parLapply.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;cl &amp;lt;- makeCluster(4) # Create the cluster
system.time(
results &amp;lt;- parLapply(cl, iter, lm_boot_fun2, data = datos_pc, n_sample = 100)
)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## user system elapsed
## 0.01 0.00 4.52&lt;/code&gt;&lt;/pre&gt;
&lt;pre class="r"&gt;&lt;code&gt;stopCluster(cl) # Close the cluster; do not forget this step&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Suppose your manager walks in and says the client wants a Random Forest—perhaps because it sounds impressive. Let us adapt the function using the randomForest package and save the pseudo-R&lt;sup&gt;2&lt;/sup&gt; and MSE.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;library(randomForest)
&lt;p&gt;rf_boot_fun &amp;lt;- function(iter, data, n_sample) {
ind &amp;lt;- sample(1:nrow(data), size = n_sample)
data_sample &amp;lt;- data[ind, ]&lt;/p&gt;
&lt;h1 id="fit-the-random-forest-model"&gt;Fit the Random Forest model&lt;/h1&gt;
&lt;p&gt;model1 &amp;lt;- randomForest(Sepal.Length ~ Petal.Length, data = data_sample)&lt;/p&gt;
&lt;h1 id="extract-the-pseudo-r2"&gt;Extract the pseudo-R2&lt;/h1&gt;
&lt;p&gt;R2 &amp;lt;- model1$rsq[length(model1$rsq)]&lt;/p&gt;
&lt;h1 id="extract-the-mse"&gt;Extract the MSE&lt;/h1&gt;
&lt;p&gt;MSE &amp;lt;- model1$mse[length(model1$mse)]
results &amp;lt;- c(&amp;quot;R2&amp;quot; = R2, &amp;quot;MSE&amp;quot; = MSE)
return(results)
}&lt;/code&gt;&lt;/pre&gt;&lt;/p&gt;
&lt;p&gt;Now let us use &lt;em&gt;parLapply&lt;/em&gt;.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;library(randomForest)
cl &amp;lt;- makeCluster(4) # Create the cluster
system.time(
results &amp;lt;- parLapply(cl, iter, rf_boot_fun, data = datos_pc, n_sample = 100)
)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Error in checkForRemoteErrors(val): 4 nodes produced errors; first error: could not find function &amp;quot;randomForest&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Timing stopped at: 0 0 0&lt;/code&gt;&lt;/pre&gt;
&lt;pre class="r"&gt;&lt;code&gt;stopCluster(cl) # Close the cluster; do not forget this step&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;strong&gt;What happened?&lt;/strong&gt; I already called &lt;em&gt;library(randomForest)&lt;/em&gt;!&lt;/p&gt;
&lt;div class="figure"&gt;
&lt;img src="https://media.giphy.com/media/DVkZv0SltX1h6/giphy.gif" /&gt;
&lt;/div&gt;
&lt;p&gt;The &lt;em&gt;randomForest&lt;/em&gt; package is loaded in our current session. However, &lt;em&gt;makeCluster&lt;/em&gt; opens &lt;em&gt;n&lt;/em&gt; new R sessions, each starting from scratch. We must load the packages in every session. In some cases, we must also export datasets and functions.&lt;/p&gt;
&lt;p&gt;To load &lt;em&gt;randomForest&lt;/em&gt; on the cluster nodes, use &lt;em&gt;clusterCall&lt;/em&gt;. It runs a function on every node; here, we use it to load the package on all of them.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;cl &amp;lt;- makeCluster(4) # Create the cluster
clusterCall(cl, function() library(randomForest))&lt;/code&gt;&lt;/pre&gt;
&lt;pre class="r"&gt;&lt;code&gt;system.time(
results &amp;lt;- parLapply(cl, iter, rf_boot_fun, data = datos_pc, n_sample = 100)
)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## user system elapsed
## 0.02 0.00 137.75&lt;/code&gt;&lt;/pre&gt;
&lt;pre class="r"&gt;&lt;code&gt;stopCluster(cl) # Close the cluster; do not forget this step&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div id="un-ejemplo-mas-realista-usando-machine-learning" class="section level2"&gt;
&lt;h2&gt;A more realistic example: machine learning&lt;/h2&gt;
&lt;p&gt;Now for a more realistic example: model &lt;em&gt;tuning&lt;/em&gt;. This usually means choosing the best value of one or more parameters according to a performance statistic, such as &lt;em&gt;R&lt;sup&gt;2&lt;/sup&gt;&lt;/em&gt; or &lt;em&gt;RMSE&lt;/em&gt;. Tuning uses the training or calibration dataset, while the model is evaluated on validation data.&lt;/p&gt;
&lt;p&gt;We will use iris as our training dataset and assume we have only sepal measurements, to make the task a little harder.&lt;/p&gt;
&lt;p&gt;We will use a &lt;em&gt;Support Vector Machine&lt;/em&gt; (SVM) to predict species from flower measurements. This model works well for classification, but can depend heavily on the cost and gamma parameters when using a radial kernel. We will try to optimise both.&lt;/p&gt;
&lt;p&gt;The &lt;strong&gt;e1071&lt;/strong&gt; package includes a &lt;em&gt;tune&lt;/em&gt; function that uses &lt;em&gt;cross-validation&lt;/em&gt;. Here we will use &lt;em&gt;10-fold&lt;/em&gt; cross-validation with three repetitions. This example comes from a real project with a large data matrix, where each model run took about an hour and a half. Without parallelisation, tuning would have taken days.&lt;/p&gt;
&lt;p&gt;Enough talking—let us look at the model!&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;library(e1071)
svm_model &amp;lt;- svm(Species ~ Sepal.Length + Sepal.Width, data = iris)
summary(svm_model)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##
## Call:
## svm(formula = Species ~ Sepal.Length + Sepal.Width, data = iris)
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: radial
## cost: 1
## gamma: 0.5
##
## Number of Support Vectors: 86
##
## ( 10 40 36 )
##
##
## Number of Classes: 3
##
## Levels:
## setosa versicolor virginica&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The default cost is 1 and gamma is 0.5. Let us examine the &lt;em&gt;performance&lt;/em&gt;.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;library(caret)
confusionMatrix(iris$Species, predict(svm_model))&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Confusion Matrix and Statistics
##
## Reference
## Prediction setosa versicolor virginica
## setosa 49 1 0
## versicolor 0 36 14
## virginica 0 12 38
##
## Overall Statistics
##
## Accuracy : 0.82
## 95% CI : (0.749, 0.8779)
## No Information Rate : 0.3467
## P-Value [Acc &amp;gt; NIR] : &amp;lt; 2.2e-16
##
## Kappa : 0.73
## Mcnemar&amp;#39;s Test P-Value : NA
##
## Statistics by Class:
##
## Class: setosa Class: versicolor Class: virginica
## Sensitivity 1.0000 0.7347 0.7308
## Specificity 0.9901 0.8614 0.8776
## Pos Pred Value 0.9800 0.7200 0.7600
## Neg Pred Value 1.0000 0.8700 0.8600
## Prevalence 0.3267 0.3267 0.3467
## Detection Rate 0.3267 0.2400 0.2533
## Detection Prevalence 0.3333 0.3333 0.3333
## Balanced Accuracy 0.9950 0.7980 0.8042&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Accuracy is 0.82. In many cases that is not enough, so let us try to improve it by tuning the parameters.&lt;/p&gt;
&lt;p&gt;We create a function that returns model performance for a given cost and gamma. Notice that I call &lt;em&gt;tune&lt;/em&gt; with only one parameter combination at a time. It could accept a list of parameters, but that would be harder to parallelise.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;tune_svm_par &amp;lt;- function(costo, gamma) {
control &amp;lt;- tune.control(
nrepeat = 3, repeat.aggregate = mean,
sampling = &amp;quot;cross&amp;quot;, sampling.aggregate = mean,
sampling.dispersion = sd,
cross = 10
)
&lt;p&gt;svm_cv &amp;lt;- tune(&amp;quot;svm&amp;quot;,
train.x = iris[, c(&amp;quot;Sepal.Length&amp;quot;, &amp;quot;Sepal.Width&amp;quot;)],
train.y = iris$Species,
kernel = &amp;quot;radial&amp;quot;,
scale = TRUE,
type = &amp;quot;C-classification&amp;quot;,
tunecontrol = control,
ranges = list(cost = costo, gamma = gamma)
)&lt;/p&gt;
&lt;p&gt;return(svm_cv$performances)
}&lt;/code&gt;&lt;/pre&gt;&lt;/p&gt;
&lt;p&gt;For parallelisation, we will use &lt;em&gt;clusterMap&lt;/em&gt;, which accepts more than one varying argument. It is the equivalent of &lt;em&gt;mapply&lt;/em&gt; in the apply family.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;costos.test &amp;lt;- seq(0.2, 4, by = 0.1)
gammas.test &amp;lt;- seq(0.2, 4, by = 0.1)
&lt;p&gt;grilla &amp;lt;- expand.grid(&amp;quot;costos&amp;quot; = costos.test, &amp;quot;gammas&amp;quot; = gammas.test)
nrow(grilla)&lt;/code&gt;&lt;/pre&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;## [1] 1521&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are 1,521 combinations. This &lt;em&gt;should&lt;/em&gt; take a while.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;t1 &amp;lt;- Sys.time()
&lt;p&gt;cl &amp;lt;- makeCluster(4) # Create the cluster
clusterCall(cl, function() library(e1071))&lt;/p&gt;
&lt;p&gt;list.errores &amp;lt;- clusterMap(
cl = cl, fun = tune_svm_par,
costo = grilla$costos,
gamma = grilla$gammas,
SIMPLIFY = FALSE, RECYCLE = FALSE
)&lt;/p&gt;
&lt;p&gt;library(dplyr)
errores.bind &amp;lt;- bind_rows(list.errores)&lt;/p&gt;
&lt;p&gt;stopCluster(cl)&lt;/p&gt;
&lt;p&gt;t2 &amp;lt;- Sys.time()&lt;/code&gt;&lt;/pre&gt;&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;print(t2 - t1)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Time difference of 52.99884 secs&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That hardly took any time! Now let us find the cost and gamma combination with the lowest error.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;knitr::kable(subset(errores.bind, errores.bind$error == min(errores.bind$error)))&lt;/code&gt;&lt;/pre&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr class="header"&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th align="right"&gt;cost&lt;/th&gt;
&lt;th align="right"&gt;gamma&lt;/th&gt;
&lt;th align="right"&gt;error&lt;/th&gt;
&lt;th align="right"&gt;dispersion&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr class="odd"&gt;
&lt;td&gt;885&lt;/td&gt;
&lt;td align="right"&gt;2.8&lt;/td&gt;
&lt;td align="right"&gt;2.4&lt;/td&gt;
&lt;td align="right"&gt;0.18&lt;/td&gt;
&lt;td align="right"&gt;0.1044681&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Let us test the SVM with those parameters.&lt;/p&gt;
&lt;pre class="r"&gt;&lt;code&gt;library(e1071)
svm_model &amp;lt;- svm(Species ~ Sepal.Length + Sepal.Width, data = iris, cost = 2.6, gamma = 2)
library(caret)
confusionMatrix(iris$Species, predict(svm_model))&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Confusion Matrix and Statistics
##
## Reference
## Prediction setosa versicolor virginica
## setosa 50 0 0
## versicolor 0 36 14
## virginica 0 10 40
##
## Overall Statistics
##
## Accuracy : 0.84
## 95% CI : (0.7714, 0.8947)
## No Information Rate : 0.36
## P-Value [Acc &amp;gt; NIR] : &amp;lt; 2.2e-16
##
## Kappa : 0.76
## Mcnemar&amp;#39;s Test P-Value : NA
##
## Statistics by Class:
##
## Class: setosa Class: versicolor Class: virginica
## Sensitivity 1.0000 0.7826 0.7407
## Specificity 1.0000 0.8654 0.8958
## Pos Pred Value 1.0000 0.7200 0.8000
## Neg Pred Value 1.0000 0.9000 0.8600
## Prevalence 0.3333 0.3067 0.3600
## Detection Rate 0.3333 0.2400 0.2667
## Detection Prevalence 0.3333 0.3333 0.3333
## Balanced Accuracy 1.0000 0.8240 0.8183&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Training accuracy increased from 0.82 to 0.84. It may not look like much, but a two-percentage-point improvement from a procedure that took less than a minute—thanks to parallelisation—is a good result.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Thank you for reading to the end of this tutorial!&lt;/strong&gt; &lt;strong&gt;I welcome your comments and questions.&lt;/strong&gt;&lt;/p&gt;
&lt;/div&gt;</description></item></channel></rss>