Python — Environment Variables Using dotenv Library

Learning how to load variables securely to your code

When you push your code to a GitHub repository or use it in production systems, there may be a need to load certain secure variables. For instance, API keys that you do not want to be visible publicly. When working on production with CI/CD systems, you may have to change parameters each time your code runs. Fortunately, there are multiple ways to accomplish this. This article will explore working with a dotenv library as one of the convenient and easy options.

Note: All code is available on my GitHub. You are more than welcome to check and use it in your projects! Also, it includes developments and code for more complex ideas in the tool, so you can find everything here.

medium-materials/dotenv-medium at main · IhorLuk/medium-materials

Getting started

To start with, you need to know that this library is not built into Python and thus, you will need to install it separately. Fortunately, it can be done easily using the following command. If you want to learn more about it, you can check out their GitHub page. Additionally, if you find it useful, it would be awesome to give this repository a star.

pip install python-dotenv

GitHub - theskumar/python-dotenv: Reads key-value pairs from a .env file and can set them as…

.env file

The second requisite for our workflow is having a .env file in the directory. This file is specifically used to store all the variables, like API keys of database credentials.

Important: when working with GitHub repositories, don’t forget to add this file to the gitignore file. Otherwise, you just load all the credentials not in the script but in a separate file, so that it will be even easier to find.

The fundamental way to read credentials from this file is to use the function load_dotenv. It doesn’t return any data but rather loads all variables to the environment and we can store them into Python variables with os the library.

It’s worth mentioning the rules of syntax for this case. According to PEP8, constant variables should be written in all capital letters with underscores separating words.

import os
from dotenv import load_dotenv

# load .env file to environment
load_dotenv()

API = os.getenv('API')
print(API)
#Test123APIkey

Let’s also take a look at the .env file. Essentially, you just need to determine each variable with a new line and use no spaces between values and equal signs.

API=Test123APIkey

Options to load from the environment

We store everything in our specially created file, as we have seen before. However, when testing the code in production, we won’t have this option. Therefore, we need to identify two other methods for loading our variables.

• One of these methods is to store the variables directly into the environment using the command line interface (terminal).

You can do it in Linux and MacOS this way:

export API="Test123APIkey"

For Windows syntax is slightly different:

set API=Test123APIkey

• The second option is to determine variables when running the script with a command line

API=Test123APIkey python main.py

It’s important to note that Python follows a specific priority order when loading environmental variables.

• Firstly, it reads the command line arguments and overwrites any variables with the same name from other options. This is especially relevant when working with multiple sources for the environment.
• Secondly, the program uses session environment variables.
• Finally, it retrieves all the environmental variables from .env file.

> python3 main.py
# Test123APIkey
> API=commandLineTest python3 main.py
# commandLineTest

Variable interpolation

The library has an interesting feature that allows for more complex identification of variables. For instance, you can use a value from one variable in another variable within the .env file. All you need to do is put the name of the variable in ${} symbols, and it will result in a complete string that you can use in the future.

API=123Test
REQUEST_STRING=api.request/${API}

import os
from dotenv import load_dotenv

# load .env file to environment
load_dotenv()

API = os.getenv('API')
REQUEST_STRING = os.getenv('REQUEST_STRING')
# print both variables
print(API, REQUEST_STRING)
# 123Test api.request/Test123APIkey

Loading variables to a dict

Storing all the variables in a dictionary, without loading them into the environment, could be necessary at times. However, I would suggest sticking to the original approach as it works perfectly fine when pushing the code to production. While it may seem useful, it’s always better to avoid unnecessary complexities. Still, this functionality exists and you can use it with dotenv_values function.

import os
from dotenv import dotenv_values

config = dotenv_values(".env")
print(config)

# OrderedDict([('API', 'Test123APIkey'), ('REQUEST_STRING', 'api.request/Test123APIkey')])

Conclusion

In this brief article, we have discussed the crucial process of storing confidential information and retrieving updated data from the environment. The python-dotenv library can be incredibly helpful when developing locally and sharing your code on platforms such as GitHub. Additionally, it can be easily implemented in production, as no modifications are required for this purpose.

I can be found on Linkedin and I am looking forward to connecting with you. Let’s engage in discussions about the intricate world of data science and data engineering.