2 posts tagged with "Linux"

When setting up a crontab job in Linux machine, these essential steps are required for a successful system operation

1. Update the cron file by adding the new script on schedule

2. Check the frequency of the schedule. Such as for running at 7 minutes interval, use */7 * * * * python /path/to/script.py Or for running every hour at 7th minute, use 7 * * * * python /path/to/script.py

3. Check the file permission for the script. If it is not executable, then make it executable ls -l /path/to/script.py

Then if the file is not executable by the user add the permission by chmod 755 /path/to/script.py (you may need to have sudo access if you are not the owner of the file. For changing ownership of a file or folder, use chown command on Linux variants)

4. Check the file permission for the output site if the script produces some files. If the continuing folder or the output file does not have a write-access, ensure it is writable. Follow the similar process as in step#3. You may channel any print statements to a file such as 7 * * * * python /path/to/script.py /path/to/output/file.txt 2>&1

Or better yet, use Python logging library to create useful log files and show if the program ran correctly.

The Problem​

Sometimes we end up zipping JSON files and putting up somewhere on the interweb. Now we need to read it back from the HTTP server and parse the file using Python. For that situation, let us assume that the zipped JSON is located at this URL:
http://example.com/python_list_turned_into.json.gz. To read this file, we need to do the following

• Fetch the file using urllib2
• Add a header which will be used to detect the gzip format
• Then open the file and read the compressed file
• Next, uncompress the file using gzip library and load it!

That's it!