Categories
Data Science

How much memory does importing PANDAS library take?

Objective Let us compare the memory consumption of Python PANDAS library. Methodology Small script called memtest.py: @profile def a(): import pandas if __name__==”__main__”: a() Test it with $python -m memory_profiler memtest.py Results: Output: The increment of memory usage in line#4 shows, it took 36.527MB to import pandas on my machine. What does your benchmarking result […]

Categories
System

Merge a repo with another as a subfolder

Sometimes we may end up with one main repository and another independently developed repository for a new feature. Later it may turn out, that the independent repo needs to become a part of the main repo as a subfolder. To do that, we can use the git command using subtree. We need to put the […]

Categories
Linux System

Best way to setup PYTHONPATH for crontab

There are many suggestions on this. Add PYTHONPATH at the end of the ~/.bash_profile or ~/.bash_login files. If they do not exist, add it to ~/.bash_profile as suggested by this StackOverflow post. export PYTHONPATH=”${PYTHONPATH}:/home/path/to/your/python/package/” But this will add the package to the current user’s PYTHONPATH. To ensure crontab gets it right away. add this line to […]