Working with CSV or TXT datasets larger than RAM in R using SQLite
Use SQLite to work with datasets that exceed available RAM, illustrated with Chile’s 2017 census microdata.

I am a data scientist and Natural Resources Engineer from Universidad de Chile, with a Master of Data Science from the University of Adelaide. I combine artificial intelligence, data science, statistical analysis and geospatial analysis to tackle environmental challenges.
Since February 2022, I have coordinated the Data Science team at Chile’s Superintendence of the Environment (SMA). I develop automation, data validation and analytical solutions for environmental monitoring and enforcement, using R, Python and SQL.
My experience also includes computer vision and deep learning for agricultural robotics, forest carbon modelling, remote sensing and survey data analysis. I have worked in Chile, Australia and Germany, combining applied research, algorithm development and interdisciplinary team coordination.
Use SQLite to work with datasets that exceed available RAM, illustrated with Chile’s 2017 census microdata.
A helper function for creating clusters on Windows and servers: the second part of the R parallelisation tutorial.
A tutorial on speeding up R workflows using multiple processor cores.
A method to predict vascular plant richness using spectral and textural variables in a heterogeneous wetland is presented. Plant richness was measured at 44 sampling plots in a …
Due to its isolated location, Chile’s flora is characterized by a large number of endemic species which has guided the decision to assign central Chile the status of a global …
Making accurate estimations of daily and annual Rs fluxes is key for understanding the carbon cycle process and projecting effects of climate change. In this study we used …
Peatlands are a type of wetland characterized by the accumulation of organic matter, called peat, and are important carbon reservoirs. In areas with poor drainage, human‐induced …