Django Models Part I - Database Setup, Django models and Django ORM

When creating a web application you will come up with a need to have some sort of mechanism to store your data permanently either on a filesystem or a Database. For this article/tutorial we will have a look inside storing your data on a Database. Database is an organized collection of data for easy write/read. There are many databases available freely to use but we will focus on RDBMS supported by Django under the hood.

Django Models

Django model is the single, definite source of information about your data
It contains the essential fields and behaviors of the data you're storing
Django model is some sort of your Database table representation a db table --> a django model
We write django models on a class based style i.e using Python Class syntax. So, Each model is a Python class that subclassed django.db.models.Model
Each attributes(valid DB type) of the model represents a database field
With all this, Django gives us an automatically generated database access API that let's us create, retrieve, update and delete objects using ORM. Thus, you hardly need to write any raw SQL. SQL stands for Structured Query Language used to communicate with a Database
This doesn't mean Django uses some sorts of magic to communicate with a Database. Under the hood Django create raw SQL for every operations(basically CRUD) to communicate with a Database
Django models lies inside models.py file on your Django apps. Each Django app can have multiple models/tables inside a models.py

Don't get confused with models.py and Model class we are talking. models.py is a python file which is inside every Django apps and Model Class is a Python class we create inside the models.py file

Basic Example of a Django Model

Lets take an example of User model which has username, first_name, last_name, attributes/field

under your models.py, let's create our first model as follows,

from django.db import models

# we need to create a class which subclasses django.db.models.Model
# so let's create a class which inherits models.Model
class User(models.Model): 
        username = models.CharField(max_length=50)
        first_name = models.CharField(max_length=30)
        last_name = models.CharField(max_length=30)

Note,

Each field is specified as class attributes, and each attributes maps to a database column
Creating attributed like age=10 will not be reflected in our Database since Django will not know how to process it and map to a database column. This means, we need to use attribute type provided by Django only. (Advanced: You can create your own fields as well, but trust me fields provided by Django will be enough for nearly all field types you will ever need)

The model class we just created will create a database table like this;

CREATE TABLE app_user(
          "id" serial NOT NULL PRIMARY KEY,
          "username" varchar(30) NOT NULL,
          "first_name" varchar(30) NOT NULL,
          "last_name" varchar(30) NOT NULL,
);

Don't get confused on how to generate this SQL, we need some commands to create a intermediate file (called migrations) which we will apply to our database (called migrate)

This is really awesome, we didn't write a single line of SQL, but we got the corresponding SQL statement to create our table. But,

we didn't add id attribute/field on our Model class, but this is included on the SQL, this is because Django will automatically add a Primary Key as id by default. Don't worry we can always customize this behavior but let's leave Django upon itself to do this good work
we created our Model class as User but the SQL seems to have created app_user table, this is because on default behavior, django will craft a table name for your automatically based on your app name and Model class name. In our case, let's assume we create a django app as app and our Model class as User, so this means django will create app_user table. This behavior can be customized by overriding the Model class Meta information

Before jumping on commands to reflect our Django Model class to our database, let's first configure our database settings so that Django can use it and do the hard work.

Databases in Django

Django officially supports
- PostgreSQL (will need PostgreSQL>=9.5 and python package psycopg2>=2.5.4). psycopg2 is a PostgreSQL adapter for Python programming language
- MariaDB (added in Django 3.0. Django supports MariaDB>=10.1)
- MySQL (MySQL>=5.7 for Django 3.2 and required python adapter mysqlclient>=1.3.12)
- Oracle (oracle>=12.12 and cx_oracle>=6.0 python driver required)
- SQlite (SQlite>=3.8.3)

SQlite is a file based database. You don't need any Python adapters to work with SQlite database, since SQlite module is integrated into python itself. This means, to work with SQlite database, just write python code using sqlite3 module. Due to this reason Django will work under the hood with SQlite Database without any special environment setups.

Since, working with SQlite initially to learn is pretty easy we will configure SQlite in our Project. Don't worry, integrating other databases is easy as well which we will focus on other articles.

By default, when creating a django project, under settings.py Django configures your Project to work with SQlite

# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}

Notes;

ENGINE refers to the corresponding database backends/code/mappings for our preferred Database

We can add multiple database configurations inside this settings. Multiple database are used for data replication, optimization, aggregating data from multiple sources e.t.c, whose configurations might look like,

DATABASES = {
  'default': {
      'ENGINE': 'django.db.backends.sqlite3',
      'NAME': BASE_DIR / 'db.sqlite3',
  },
 'read_replica': {
      'ENGINE': 'django.db.backends.sqlite3',
      'NAME': BASE_DIR / 'db_replica.sqlite3',
  },
  'analytics_db': {
      'ENGINE': 'django.db.backends.sqlite3',
      'NAME': BASE_DIR / 'analytics_db.sqlite3',
  }
}

When connecting to other databases special/extra connection parameters will be required, let's take an example of PostgreSQL

DATABASES = {
  'default': {
      'ENGINE': 'django.db.backends.postgresql',
      'NAME': 'db_name',
      'USER': 'db_user',
      'PASSWORD': 'db_password',
      'HOST': '127.0.0.1',
      'PORT': '5432',
  }
}

So we checked our configurations and now let's head to the actual process of reflecting our Model Class to our Database as a Database table

Django Migrations (Migration framework)

Migrations are Django's way of propagating changed you make to your models into your database table
available commands:
- python manage.py makemigrations responsible for creating new migrations based on changes you made to your models. This is an intermediate Python file django created before taking changes to the actual database server. Think of migration file as a history tracker (GIT) for your models. You create migrations after you make any changes to the models which should reflect to your database. Migrations files are created and placed under your apps migrations folder
- python manage.py migrate responsible to pick the created migrations files and to peform apply/unapply operations which gets reflected to your database.
- python manage.py sqlmigrate [app_name] [migration_file_name] which displays SQL statements for the migrations created using makemigrations command
- python manage.py showmigrations which lists project's migrations and their status (i.e if it is applied to database or not)

So, the rule of thumb is first your create your migrations by using python manage.py makemigrations command and then apply those changes to database with python manage.py migrate

Reminder: Did you perform makemigrations and migrate command?

Model Field Options

Model field options are the options we can pass to individual model attribute classes. In previous example when we used models.CharField we used max_length option to tell django that the field should only have data lenght of upto given max_length. There are more options that we can pass to our model attributes depending upon our requirements, some of commonly used model options are:

null(default=False) when not provided it defaults to False, i.e: null value is not allowed
blank(default=False)
db_column used to create/customize column name on the Database table which if not provided defaults to the attribute name
default (used to provide default value to the attribute incase a value is not provided when adding a new object/row to a model/table). default can be a constant value or a callable object like python functions, this python function should return a value which can be used a default value for the attribute
unique(default=False) if true the field value must be unique throughout the table
validators a list of validators to run for this field

A simple example of Model using different model options

from django.db import models

def dummy_default_extra_data():
    return "Additional Information and data"

class DummyModel(models.Model):
    first_name = models.CharField(max_length=50)

    # assume, middle_name can be null
    middle_name = models.CharField(max_length=50, blank=True, null=True)

    last_name = models.CharField(max_length=50)

    # date_of_birth is long so create a dob for column name,
    # but, use date_of_birth inside django
    date_of_birth = models.DateTimeField(db_column='dob')

    # assume, email_notification is allowed by default
    email_notification = models.BooleanField(default=True)

    # extra_data default as a function,
    # (you can't use lambda function here)
    extra_data = models.TextField(default=dummy_default_extra_data)

    # assume, passport_number should be unique
    passport_number = models.CharField(max_length=100,unique=True)

ORM (Object Relational Mapper)

ORM Helps us write queries(simple or advanced) using object oriented paradigm of your favourite programming language
- Django ORM helps us to communicate with our database without using any SQL commands.
Abstract away the database system, so switching from MySQL to PostgreSQL is pretty easy

For now, to practice ORM(communicate with DB) we will not create any views or any complicated python code, instead we will use a shell provided by Django to run our ORM code. This django shell is an advance version of a Normal Python shell but Django modules are already loaded inside this shell.

You can open django shell using command python manage.py shell

For now, lets assume a simple model, we will use this model to understand ORM code

from django.db import models

class Information(models.Model):
        name = models.CharField(max_length=50)
        age = models.IntegerField()
        is_active = models.BooleanField(default=False)

Once, you have created your models django provides you with a default Manager called objects through which you can you can perform database operations. You can always customize or add more Managers to your models which we will see in future articles.

Don't forget, we will be getting familiar with ORM inside django shell for now. So open up the django shell

Creating Objects/ Adding row to DB table

save() method

   # assuming Information model belongs to practice app 
   from practice.models import Information 

   obj = Information()
   obj.name = "John"
   obj.age = 15
   obj.save()

create() method

  # assuming Information model belongs to practice app 
  from practice.models import Information 

  obj = Information.objects.create(name = "John")

Retrieving objects / get row/s from DB table

get() a single row. Can return/work with only single row/object

  # assuming Information model belongs to practice app 
  from practice.models import Information

  john_obj =  Information.objects.get(name="John")

  # will throw error if no row matched or multiple row is matched. 

  # get a object with non existing row to see the error
  check_error = Information.objects.get(name="Nothing")

all() to retrieve all rows from table, which returns an iterable queryset which can be looped to retrive every objects

  # assuming Information model belongs to practice app 
  from practice.models import Information

  queryset = Information.objects.all() # actual ORM

  for obj in queryset:
        print(obj, obj.name, obj.age)

filter() to retrive multiple rows (a queryset) but with certain filters/ SQL where statement

 # assuming Information model belongs to practice app 
 from practice.models import Information

 # get all objects/rows with age=15
 queryset = Information.objects.filter(age=15)

  for obj in queryset:
        print(obj, obj.name, obj.age)

Updating objects/ update rows of DB table

single update/ single row update

     # first get the object which you want to update

     # lets get the John and update his age to 20
     obj = Information.objects.get(name="John")
     obj.age = 20 # update age
     obj.save()

Multiple objects/rows update at once

   # lets update every rows to is_active=True 
   updated = Information.objects.all().update(is_active=True)
   print(updated)

Deleting a object/ a row from DB table

Single object delete/ single row delete

    # first get a object, and delete

    # lets delete John
    john_obj = Information.objects.get(name="John")
    john_obj.delete()

Multiple objects/rows delete at once

 # delete every objects from DB
 deleted = Information.objects.all().delete()

 # delete every rows whose is_active=False
 deleted = Information.objects.filter(is_active=False).delete()

Raw SQL

When working sometimes you may need to perform raw SQL instead of the ORM. This can come up when you need to write advanced SQL, or maybe use some native DB feature which Django ORM doesn't support or maybe the raw SQL is pretty fast than the ORM, whatever can be the reason and its really accepted to write raw SQL as well, but i suggest to stick to ORM as much as possible.

raw() which maps to models attributes automatically

 data = Information.objects.raw(
                "SELECT id,name,age FROM practice_information"
             )

You can also perform SQL directly,

    from django.db import connection

    with connection.cursor() as cursor:
             cursor.execute("SELECT * FROM practice_information;")
             rows = cursor.fetchall()
             print(rows)

Whats next?

In the next article we will see more about the models relations/DB table relationship and advance guides on lookups and QuerySet

Django Models Part I (Database Setup, Models and ORM)