How to deploy a Python Flask App with NGINX and Gunicorn

Preface

Deploying a Flask web application is an essential step in making your project accessible to users worldwide. This blog post will guide you through the process of deploying a Flask app with NGINX and Gunicorn using python3 virtual environments (venv), Supervisor for process management, custom DNS for domain configuration, and Git for version control. While the steps may involve technical details, the guide is designed to be (almost) beginner-friendly. You can also get the updated version of this post in my GitHub.

Tech Stack

For this sample project, I am going to use:

• Ubuntu 22.04: for development, test and deployment.
• Git & GitHub: For version control and repositories hosting services.
• Python3: for scripting and our primary language.
• Flask: A python based web framework.
• IDE: of your choice. Personally I use VS Code but does not matter if you prefer something else.
• NGINX: A powerful and advanced load balancer, Web server and reverse proxy.
• Gunicorn: or ‘Green Unicorn’ is a Python WSGI HTTP Server for UNIX or in our case Ubuntu.
• Supervisor: is a process manager to monitor and automate Gunicorn to starts, restarts, and runs when we need it to.

Prerequisites & Requirements

Non-technical:

We need to be resilient. Cause, depending on one’s experience, this project can be somewhat challenging. But don’t you worry. Through grit and determination, we will see to the end. We will encounter many bugs, but we will squash them through research, learning and going in depth.

Technical:

# Basic Knowledge of Web Development and Deployment:

• Understanding of HTML, CSS, and basic web development concepts.
• Familiarity with deploying web applications and basic server management.

# UNIX based OS for development:

• Preferably Ubuntu, but macOS are fine too. As system directories are different on Windows 🙁

# Basic knowledge about python and flask

# sudo or root user privileges for Ubuntu

All the other techs, we will learn as we move forward if you don’t know them already!

Note: I did not set up any git repo or not going to. The idea is the learner will learn from scratch without copy-pasting or cloning any code. But if you still like to see some documentation, then please go to this repo of mine for reference.

Project LifeCycle

Step 1: Project Setup for local development

• Initiate python virtual environment and activate
• Initiate Git
• Add GitHub Repo
• Add .gitignore
• First Git stage, commit and push

then test, test, test!

Step 2: Basic PIP installations and codebase for the app

• Flask
• Gunicorn
• etc

then test, test, test!

Step 3: where’s the magic happens..

Part 1: Server Configuration

Part 2: SSH key based authentication setup for remote server

then test, test, test!

Step 4: Configure remote host firewall

then test, test, test!

Step 5: Test and deploy the flask app

Step 6: Production Server and final touches

Conculasuion

Enough chat. Let’s dive..

Step 1.1: Initiate python virtual environment

First thing first, let’s initiate a virtual environment for our sample project. It is important and highly recommended using a virtual environment for local development and testing. Otherwise, all of our dependencies will be installed globally, which is not recommended at all! Virtual environments are isolated environments that allow you to manage project-specific dependencies without affecting the system-wide Python installation. This is a common practice in Python development to keep dependencies for different projects separate and avoid conflicts.

Let’s say our sample project directory name is “sample-flask-app” (without quotes ” “). Now go to this directory and open it in terminal. Once the terminal is open check if you are in the directory write the command:

python3 -m venv venv

The command python3 -m venv venv is used to create a virtual environment for a Python project. Let’s break down the components of this command:

python3:

• This specifies the Python interpreter to use. In this case, it’s Python 3. If you have both Python 2 and Python 3 installed on your system, using python3 ensures that the virtual environment is created for Python 3.

-m venv:

• The -m flag allows running a module as a script. venv is a built-in Python module used for creating virtual environments. The combination -m venv means you’re running the venv module to perform virtual environment-related actions.

venv:

• This is the name of the virtual environment directory that will be created. In this case, it’s named venv by convention, but you can choose a different name if you prefer.

So, when you run python3 -m venv venv, you’re telling Python to use the venv module to create a virtual environment named venv in the current directory.

Note: Just in case if you don’t have venv module installed in your system then please command in your terminal:

sudo apt-get install python3-venv

this command will install venv module in your system. However, this can be installed systemwide if you are wondering. To know more about virtual environment, you can follow this link

after that please do

python3 -m venv venv

again and in this point you should see a folder name venv in your project directory. yay!

Now let’s go and activate this venv(virtual environment) by the command in our terminal:

source venv/bin/activate

Ok what we just did? The command source venv/bin/activate is used to activate the virtual environment you created using python3 -m venv venv. Let’s break down what this command does:

source:

• source is a command in Unix-like operating systems (including Linux and macOS) that reads and executes commands from a file, in this case, the activate script within the venv/bin/ directory.

venv/bin/activate:

• This is the path to the activation script for the virtual environment. When you run source venv/bin/activate, it executes the commands in this script to modify the shell environment.

When you activate a virtual environment, it does a few things:

• It modifies the PATH environment variable to include the bin directory of the virtual environment, ensuring that when you run Python or other commands, it uses the versions and packages within the virtual environment.
• It sets the terminal (the shell prompt) to show the name of the virtual environment, making it clear which environment is currently active.
• It may perform other environment-specific configurations.

Once you’ve activated the virtual environment using source venv/bin/activate, you’ll see the virtual environment name in your shell prompt, indicating that you are now working within the isolated environment. This allows you to install and manage project-specific dependencies without affecting the system-wide Python installation.

From now on, every time we work on this project, we will have to activate the venv first!

Important note! Once you created the venv it is recommended to not move or rename the project directory otherwise it won’t work and will face complicacy!

Let’s go next step..

Step 1.2: Set Up Version Control with Git

Now in the same project directory name “sample-flask-app” initiate git to keep track our development process and progress. Git is version control, which helps manage your codebase efficiently. A must-know for a developer.

git init

Consider creating a .gitignore file in the same project directory to exclude unnecessary files (e.g., virtual environment, cache files) from version control etc. Here is a sample .gitignore file for you. just copy the content and save it in the name .gitignore

If you like to host/push your code to GitHub, then please create a repository there and follow the step-by-step guide. Once you are ready, then simply push the code to GitHub. Please follow the official documentation about how to use Git and GitHub

At this point our project setup in local environment is done, we can move onto some coding 🙂

Building / Development Phase

Step 2: Install Flask and Other Dependencies:

At this phase, you should have / build your Flask application according to your needs. Starting with installing dependencies etc.

Example:

pip install Flask

Ensure to install any other necessary dependencies you need for your project. Once development is done, then move onto next phase, Deployment!

Deployment Phase

Step 3 Part 1: Server Configuration using Nginx-gUnicorn for Linux (first time)

step 3.1: at first open up local ssh shell/ unix terminal

from now on I will use cmd: as a shorthand for command in the bash shell. If you copy command from here please copy without cmd: and use appropriate usernames, directory, path etc according to your project!

step 3.2: login to remote Linux server via ssh

cmd: ssh username@remote_host_ip –> then type password on prompt step

step 3.3: after logging in, (assuming this cloud server is fresh and nothing in it!) do a system wide update and upgrade

cmd: apt update && apt upgrade (first time recommended)

cmd: sudo apt update

step 3.4: set hostname

cmd: hostnamectl set-hostname [hostname ie: flask-server]

note: type hostname to double check what hostname is created

step 3.5: add hostname to the host file first open host file using nano

cmd: nano /etc/hosts

then under the localhost IP(127.0.0.1) in new line write VPS or dedicated server IP address provided by cloud provider and newly_created_hostname. like 00.00.00.1 your_newly_created_hostname ie: flask-server then ctrl+x to back out and save on prompt and then ctrl+x to back in the shell again

step 3.6: add a limited user with sudo privilege (currently we are on root!)

cmd: adduser [username] –> then type a strong password on prompt followed by optional information

#after that add newly created user to the sudo group
cmd: adduser [username] sudo

#now log back in the server with newly created sudo user
cmd: exit (to logout from the server)

#then log back in with new sudo user
cmd: ssh [new_sudo_user]@[ip_address], 

#input the new user [password] on prompt

Step 3 Part 2: SSH key based authentication setup for remote server

“”” This step is optinal but necessary for security, against brute force attack and also saves time “””

            ### Manual steps(recommended): for linux local machines ###

step 3.7: create private key (for local machine) and public key (remote or even virtual machine)

generate ssh key (local machine terminal):

cmd: ssh-keygen -t rsa -b 4096 –> key will generate with a fingerprint id and a random image in local default ~/.ssh directory

now if we cd into cmd: cd ~/.ssh directory and cmd: ls -la we will see id_rsa private key and id_rsa.pub as public key

step 3.8: push ssh public key to remote sever

first check if we are in remote server’s home dircectory by

cmd: pwd or just cmd: cd to go to home directory

then make new .ssh directory in remote home directory by

cmd: mkdir .ssh

step 3.9: now we are gonna push our locally created ssh public key to remote .ssh directory from local .ssh directory.
**we are gonna use scp command which allows us to copy file from our local terminal.
** first we need to make sure we are in .ssh directory in our local machine
then type cmd below in our local machine
cmd: scp ~/.ssh/id_rsa.pub [user_name]@[remote_ip]:/home/[username]/.ssh/[uploaded_key].pub
** make sure we check our cmd above precisely and according to our own usernames and directories

now if we check in our remote connected shell by cmd: ls .ssh/ we will see uploaded_key.pub is created and uploaded

step 3.10: append the newly uploaded key as authorised key file

cmd: cat ~/.ssh/uploaded_key.pub >> ~/.ssh/authorised_keys

** command above will append/add the public key to authorized_keys file(no file extension needed for unix servers)

** now if we cmd: cat ~/.ssh/authorised_keys we will see the public key that just added to the authorised_key file

step 3.11: now we need to setup correct permission for our public key .ssh folder(code=700) and authorized_key file(code=600)

first we set permission 700 to ~/.ssh/ folder by cmd: chmod 700 ~/.ssh/ (this will give the folder necessary 700 permission to .ssh folder)

–> then we will set permission 600 to all the files in .ssh folder by
cmd: chmod 600 ~/.ssh/*

“”” now we should be able to login from local to remote system without password. To double chek logout from remote machine by cmd: exit and log back in with only ssh [username]@[remote_ip] We should be logged in without any password.

## additional steps for maximum security and force all user to log in with ssh key and not with any password on remote server ##
    
    ### *** important: Changing config file incorrectly might cause fatal errors in the server!!! *** ###
    
    
** First we need to backup remote ssh config file for obvious reason**
    backing up ssh config file by copying it to same folder
    cmd: sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak
        --> command above will make a backup of config file

    now change the config file with nano editor
    cmd: sudo nano /etc/ssh/sshd_config
    
** after that config file will open in nano editor and we need to find     PasswordAuthentication and change it from yes to no
        and uncomment it if it was commented via deleting # sign in front. then ctrl+x and save it. 

        note: we can add PasswordAuthentication no line if it isnt there! ** 
        
    now we need to restart the ssh service to activate the changes by command
    cmd: sudo service ssh restart

### Easy steps(lazy way): for linux local machines or for Mac systems via Homebrew package manager ###

 * for local Mac system install homebrew and then go .ssh folder and 
cmd: brew install ssh-copy-id --> to access the ssh key gen 

** then cmd: ssh-copy-id [username]@[remote_ip] to copy the public key 

*** then ssh into remote machine ssh [username]@[remote_ip] and we should be able to log in without making any ssh file in local machine or password 

**** we still need to manually disable PasswordAuthentication to no

Step 4: Configure remote host firewall

step 4.1: we are gonna install ufw or “uncomlicated firewall in remote host

cmd: sudo apt install ufw

–> to check if ufw is active after installation

cmd: sudo ufw status

–> if not active, enable by

cmd: sudo ufw enable

now we can see the status of our firewall rules and manipulate them to our security need.

step 4.2: now we need to configure firewall rules

cmd: sudo ufw default allow outgoing

cmd: sudo ufw dafault deny incoming

cmd: sudo ufw allow ssh

*** for setting up development test server port 5000

cmd: sudo ufw allow 5000

–> now development port 5000 is active where we will be able to see our app live in next phase but keep in mind it’s for testing purpuse only!

Step 5: Test and deploy the flask app

step 5.2: Get the environment ready.

If your local app is in a virtual environment(which is highly recommended) then we need to make a requirements.txt file but before that, we can check all the dependencies are there in our virtual environment. To do that type the following command in an active virtual environment

      cmd: pip freeze
      --> this command will show us all the dependencies we are using in our local virtual environment. again make sure you are in your plocal project's virtual environment and activated (venv)

step 5.3: To make an requirements.txt file in local Linux/Mac environment

cmd: pip freeze > requirements.txt
--> this command will make a requirements.txt file in our local project directory
(IMPORTANT: We need to make sure our our virtual envrionment is active otherwise all the Global dependencies/libraries will show up in requiremnets.txt)

step 5.4: now we need to upload all the files from local virtual environment to remote server home directory.

   There are two ways to do it:
        #1: through SSH CLI using command
            cmd: scp -r local_dircetory_project_folder remote_cloud_user@remote_IP_address:~/ (~/ remote user home folder)

            example: scp -r Desktop/Project_Folder tusar@45.33.123.214:~/
            --> this command will copy everything local to remote folder. check th ls command in remote server if project directory copied as intended

        #2: through FTP/SFTP protocol like Filezilla
            for this we need remote host user name, port(22), password. Please see your particular cloud hosting services configuration, can be AWS, Linode, IONOS etc

step 5.5: now in the remote server’s ~/ (user directory) we need to install python3 and pip (not venv yet, it’s on next step. also using python3 usually not pyhon2)

#on your remote linux server

cmd: sudo apt install python3-pip

step 5.5: now install virtual environment or venv on remote ~/ server location

cmd: sudo apt install python3-venv

step 5.6: create venv on remote server

cmd: python3 -m venv 'venv_name'

(I always use `venv` as name)
example: python3 -m venv venv 

(in remote project directory) check ls command in directory to see if venv created

step 5.7: activate virtual environment in remote

cmd: source venv/bin/activate

–> virtual environmanet should be activated now. we will see (venv) at the left side of our CLI prompt if it is activated

–> also we need to make sure we wre on our right directory(where project root files are like app.py, requirements.txt etc we can check it again by ls command)

step 5.8: now lets install all the dependencies from the uploaded requirements.txt

cmd: pip install -r requirements.txt 
--> this will install all the packages/dependencies we need for the project

Now lets test the remote server

step 5.9: lets see what is in our main_app.py file has by cmd: cat app_name.py

–> this will show us the file content in .py file.

NOTE: this is a nice trick to see quickly the file contents which is hugely useful in times

after that we need to test the server by temporarily exporting the project to see from outside world.

cmd: export FLASK_APP=run.py
cmd: flask run --host=0.0.0.0 

note: both of those command has to be exactly as showed here. if everything is correct this commands will spin up the server in http://0.0.0.0:5000/

now if we go to our ip_address:5000 we will see our project is working

step 5.10: now let’s install NGINX and gUnicorn

/ we need to make sure we are still in our virtual environment while we do these

cmd: sudo apt install nginx –> is gonna handle our web server like static files etc
cmd: pip install gunicorn –> is gonna serve our python scripts

step 5.11: now let’s confugure nginx

/ first we need to remove default nginx sites-enabled config file. usually the directory for it is: /etc/nginx/sites-enabled/default we can also check it before we remove it by cd into that directory

cmd: sudo rm /etc/nginx/sites-enabled/default

/ now lets create a new config file

cmd: sudo nano /etc/nginx/sites-enabled/pick_any_name

/ in this config file we define our server and static file locations for nginx as follows

    server {
            listen 80;
            server_name cloud/our_ip_iddress;

            location /static {
                alias /project_path/static_folder_directory; --> we can also use root instead of alias but in some specific cases. also writing with forward slash / after is recommended in some cases. like /static/
            }

            location / {                          --> this will handover http requests to gunicorn
                proxy_pass http://localhost:8000; 
                include /etc/nginx/proxy_params;
                proxy_redirect off;
            }
    }

    

Since Nginx is handling the static files directly, it needs access to the appropriate directories. 
    We need to give it executable permissions for our home directory. The safest way to do this is to add the Nginx user to our own user group. 
    We can then add the executable permission to the group owners of our home directory, giving just enough access for Nginx to serve the files:

        cmd: sudo usermod -a -G your_user www-data 

        --> this will add the Nginx user to our own user group
        --> example: sudo usermod -a -G tusar www-data

        cmd: sudo chown -R :www-data /path/to/your/static/folder

        --> this will give Nginx correct permission to read and serve the static files

        also I needed to fix site-enabled section in etc/nginx file to setup the location of static file  

step 5.12: now we can update the ufw ports after our testing is done on port: 5000 and serve our app from port: 80

/ first allow port 80
cmd: sudo ufw allow 80

/ then disable port 5000
cmd ufw delete allow 5000

/ the enable/update the new ufw rules
cmd: sudo ufw enable

// then restart the nginx server
cmd: sudo systemctl restart nginx --> this will start nginx server but not gunicorn! next step ¬

step 5.13: now lets run gunicorn

cmd: gunicorn -w 3 app:app

// ok so here is the explanation of it. 

gunicorn = gunicorn server (duh!)
-w = workers 
    --> sounds prehistoric but thats how it is. simple but vauge :( so in gunicorn architecture it means that there is a central master process that manages a set of worker processes. 
        The master never knows anything about individual clients. All requests and responses are handled completely by worker processes.
        to know more plz visit: https://docs.gunicorn.org/en/stable/design.html#:~:text=Gunicorn%20is%20based%20on%20the,handled%20completely%20by%20worker%20processes.

3 = worker numbers
    --> the way we need to calculate the worker numbers are: 
        -w = (2 x number of our cloud virtual machine core) + 1
          so -w = 3 (in my case)

          --> to find out yours just do:
              cmd: nproc --all

app: = name of our main script like app.py/main.py etc in my case app.py

:app = the variable name that we used to create the app. like

    app = create_app()

/ vola! now everythin is working! but it’s not it! cause we are still in developments and not final production mode on our remote server!

Step 6: Production Server and final touches

step 6.1: now we need to configure gunicorn to auto start and restart. we were doing manually thus far. we are gonna use a software called “SUPERVISOR” for this job

cmd: sudo apt install supervisor

step 6.2: now we need to setup a configuration file for supervisor

cmd: sudo nano /etc/supervisor/conf.d/give_it_a_file_name.conf 
--> this will create a blank config file

step 6.3: in the newly created config file, we need to declare few things like

    [program:give_any_name]
    directory=project_directory/
    command=project_directory/venv/bin/gunicorn -w 3 app:app
    user=user_name
    autostart=true
    autorestart=true
    stopasgroup=true
    killasgroup=true
    
    #for our error logs
    stderr_logfile=/var/log/pick_a_relatabel_name/error_file.err.log 
    stdout_logfile=/var/log/same_name_as_last_line/error_file.out.log

step 6.4: now lets create directory for log files

cmd: sudo mkdir -p /var/log/same_name_as_var_log_file 

/ then make error files to this directory we defined earlier in supervisor config file

cmd: sudo touch /var/log/same_directory_we_created_last_command/error_file.err.log
cmd: sudo touch /var/log/same_directory_as_last_command/error_file.out.log

!DEBUG: please check carefully that defined logfile links and directory is the same as we created. easy mistake can be made here 

/ then restatrt the supervisor

cmd: sudo supervisorctl reload

/ now check the app if its running in the server! mine does.. YEEEEHHHAAA
/ to be more precise, restart the virtual machine (sudo reboot) and see if it's still running. give it couple of minutes to reboot.

That’s All!

Conculasuion

Appologise if this has been a long winded article to follow. But it is absolutely imperitive to follow industry best standards for deployment of a real world reliable, scalable and maintainable application.

By following those steps, you have successfully deployed your Flask app using NGINX and Gunicorn, managed by Supervisor, with version control using Git, and a custom DNS for accessibility.

Let’s have a discussion if any of my provided code does not work or for anything else related.

I had to research a lot during the writing process and learned a lot of stuff and absolutely loved it!

A big big thanks goes to brilliant Corey Schafer and his amazing youtube channel! Please check out his impressive contents.

Happy learning 🙂