Learn Django in 14 days - Day 2: Models and Databases

Models in Django are used to define the structure and behavior of data stored in the database. Understanding how to work with models is essential for building robust and scalable web applications. So, let's dive into the world of Django models!

The sample source code is added here

1. Introduction to Django databases and models

Django's database support is based on its powerful Object-Relational Mapping (ORM) layer. The ORM abstracts away the complexities of interacting with different database management systems (DBMS) and allows developers to work with databases using Python code instead of writing raw SQL queries (which is NOT recommended in anyway).

Django supports various relational databases, including PostgreSQL, MySQL, SQLite, and Oracle, among others. This flexibility enables developers to choose the most suitable database backend for their project's requirements. The recommended Django database is PostgreSQL.

In which, a database model is a Python class that represents a database table. It defines the fields and methods for interacting with the underlying database. Django models provide a high-level and intuitive way to work with databases, making it easier to manage and manipulate data.

django model

Then, we will discuss some advance topics with additional and helpful packages

Fields
- https://github.com/respondcreate/django-versatileimagefield: An rich-features ImageField package
- https://github.com/romgar/django-dirtyfields: Keep trach of model instance changes and the current stored database values
- https://github.com/stefanfoulis/django-phonenumber-field: Python Phone Number validation fields
Models
Database Config
- https://github.com/jazzband/dj-database-url

2. Creating a Django model

To create a Django model, you need to define a Python class that inherits from the django.db.models.Model base class. This class represents a table in the database. Each attribute of the class represents a field in the table.

Defining fields

Fields define the type of data that can be stored in the database. Django provides various field types such as CharField, IntegerField, DateField, ForeignKey, and many more. You can choose the appropriate field type based on the data you want to store.

Field types

Django offers a wide range of field types to handle different types of data. For example, CharField is used for storing text, IntegerField for storing integers, DateField for storing dates, and so on. Choosing the correct field type ensures data integrity and efficient storage.

Field options

Fields can have additional options to customize their behavior. Some common options include null to specify if a field can be empty or not, default to provide a default value for the field, unique to enforce uniqueness, and many more. These options allow you to fine-tune the behavior of your models.

An example of a model is defined as below

from django.contrib.postgres.fields import ArrayField

from django.db import models





class Book(models.Model):

    author = models.ForeignKey(

        "Author",

        on_delete=models.CASCADE,

        related_name="jobs",

    )

    tags = ArrayField(

        models.CharField(max_length=256, null=True, blank=True),

        blank=True,

        null=True,

    )

    description = models.TextField()

    title = models.CharField(max_length=256, null=True, blank=True)

    category = models.CharField(max_length=256, null=True, blank=True)

    price = models.DecimalField(max_digits=None, decimal_places=None)

    

    def is_comic(self):

        return "comic" in self.category.lower()

3. Database migrations

When you make changes to your models, such as adding or modifying fields, Django provides a migration system to manage these changes in the database schema.

Generating migrations

Django's migration system allows you to automatically generate the necessary SQL statements to apply your model changes. You can use the makemigrations command to generate migration files based on the changes detected in your models.

python manage.py makemigrations app

python manage.py makemigrations

django model

Applying migrations

Once you have the migration files, you can apply them to the database using the migrate command. This will update the database schema to match the current state of your models. Django keeps track of which migrations have been applied in the table django_migrations in the current database, making it easy to manage schema updates.

python manage.py migrate <app> <migration_number></migration_number></app>

django model

Rollbacks and reversions

Django also supports rollbacks and reversions of migrations. If you encounter issues after applying a migration, you can roll it back to restore the previous state. Additionally, you can revert a set of migrations to go back to a specific point in time.

python manage.py migrate <app> <migration_number>

python manage.py migrate bookstore 0001

python manage.py migrate bookstore zero  # Revert it back to zero</migration_number></app>

Fake migrations

This is an important feature when there are some conflicts between the actual database updates and the migration number stored in django_migrations table. Imagining a case when you already made changes in the databases due to an urgent request from client, but you haven't created a migration script and applied it to the database. This is when the --fake migration is used.

python manage.py migrate bookstore 0002 --fake  # the current migration number is 0001, after this change, it will be 0002 without an actual migration performance.

Pros

Database Schema Evolution: This provides a seamless and controlled evolution of the database schema over time. It simplifies the process of making changes to the database structure without requiring manual SQL scripts or recreating the entire database.
Version Control: Migrations are stored as versioned files in the project's codebase, making it easy to track and manage changes to the database schema alongside the application's source code. This facilitates collaboration, code reviews, and rollback if necessary.
Data Migrations: In addition to structural changes, Django migrations support data migrations. This allows you to write Python code to migrate and transform data when making schema changes, ensuring data integrity during the migration process.
Dependency Management: Migrations support dependencies between different migration files. This means you can define the order in which migrations should be applied, handling complex scenarios where one migration relies on the completion of another.
Rollback and Revert: This allows you to do/undo changes to the database schema. This can be helpful in cases where a migration introduces an issue or when you need to roll back to a previous state.

Cons

Complex Migrations: In certain scenarios, especially when dealing with complex database changes, writing and managing migrations can become challenging. Handling edge cases, such as renaming or altering existing fields, may require manual intervention or custom migration scripts. Some custom migrations can be broken after a few years due to many database constraints updates.
Performance Impact: Migrations can impact application performance during the migration process, particularly when dealing with large datasets. Applying complex migrations or migrating large amounts of data may require careful consideration and optimization to minimize downtime and performance degradation. There is a tip to avoid that by performance migration in background tasks (Celery + Redis)
Interactions with Existing Data: Migrating existing data can be tricky, especially when dealing with data transformations or data migrations that require complex logic. Ensuring data integrity and consistency during the migration process may require careful planning and testing.
External Dependencies: Migrations may rely on external dependencies or factors outside the migration system itself. Changes to external systems, such as database servers or libraries, can sometimes introduce compatibility issues and require additional effort to resolve.

4. Querying the database

Django provides a powerful ORM (Object-Relational Mapping) that allows you to query the database using Python code. The ORM abstracts the underlying database engine, making it easy to write database queries in a database-agnostic manner.

Basic queries

You can retrieve objects from the database using the objects attribute of a model class. For example, MyModel.objects.all() returns all objects of the MyModel class. You can also filter objects based on specific criteria using the filter() method.

Book.objects.filter(author_id__in=[1, 2, 3])  # This returns all books with author id as 1,2,3

Book.objects.filter(author__name__contains="Joe", category__icontains="comic")  # This returns all books with author name containing "Joe" and category containing "comic" case insensitive.

Filtering and ordering

Django provides various methods for filtering and ordering query results. You can use methods like filter(), exclude(), and order_by() to narrow down the results and specify the desired ordering.

Book.objects.filter(author__name__contains="Joe", category__icontains="comic").exclude(author__name__contains="Vu").order_by("-id")  # This returns all books with author name containing "Joe" and category containing "comic" case insensitive, but exclude those books with author name containing "Vu" case sensitive. The returned list is ordered by "ID" desc.

Aggregation and annotation

Django's ORM also supports aggregation and annotation to perform calculations and aggregations on query results. You can use methods like count(), sum(), avg(), and annotate() to retrieve aggregated data from the database.

Book.objects.aggregate(count=Count("id"))

Book.objects.aggregate(total_price=Sum("price"))

5. Relationships between models

Django allows you to define relationships between models, which enables you to establish connections and associations between different entities in your application.

One-to-one relationship

A one-to-one relationship is a common type of relationship where each record in one model is associated with exactly one record in another model. You can define a one-to-one relationship using the OneToOneField field type.

One-to-many relationship

A one-to-many relationship represents a relationship where one record in a model can be associated with multiple records in another model. This is achieved using the ForeignKey field type.

Many-to-many relationship

A many-to-many relationship is a more complex relationship where multiple records in one model can be associated with multiple records in another model. Django provides the ManyToManyField field type to handle this type of relationship.

6. Model inheritance

Django supports model inheritance, allowing you to create more specialized models based on existing models.

Abstract base classes

You can define an abstract base class as a parent for other models. Abstract base classes are used solely for inheritance purposes and are not created as separate tables in the database.

from django.db import models





class BaseModel(models.Model):

    created_at = models.DateTimeField(auto_now_add=True)

    updated_at = models.DateTimeField(auto_now=True)



    class Meta:

        abstract = True



    @property

    def added(self):

        return self.created_at



    @property

    def updated(self):

        return self.updated_at



    @property

    def created_by(self):

        if hasattr(self, "added_by"):

            return self.added_by

        return None

Multi-table inheritance

With multi-table inheritance, each model in the inheritance hierarchy is stored as a separate table in the database. Django automatically creates relationships between these tables to maintain the inheritance structure.

Proxy models

Proxy models are another form of model inheritance where you can create a proxy model that behaves like the original model but with some modifications. Proxy models are useful for adding extra methods or changing the default behavior of a model.

from django.contrib.postgres.fields import ArrayField

from django.db import models





class Book(models.Model):

    author = models.ForeignKey(

        "Author",

        on_delete=models.CASCADE,

        related_name="jobs",

    )

    tags = ArrayField(

        models.CharField(max_length=256, null=True, blank=True),

        blank=True,

        null=True,

    )

    description = models.TextField()

    title = models.CharField(max_length=256, null=True, blank=True)

    category = models.CharField(max_length=256, null=True, blank=True)

    price = models.DecimalField(max_digits=None, decimal_places=None)

    

    def is_comic(self):

        return "comic" in self.category.lower()





class KidBook(Book):

    class Meta:

        proxy = True



    def for_kid(self):

        return True

7. Best practices for Django models

Naming Conventions
- Use singular nouns for model class names, and make them descriptive and intuitive.
- Use lowercase letters and underscores for field names.
- Consider using verbose and meaningful names for fields to improve code readability.
Field Types and Constraints
- Choose the appropriate field types that accurately represent the data being stored.
- Add constraints such as null=True, blank=True, and unique=True to enforce data integrity.
- Use ForeignKey and ManyToManyField to establish relationships between models.
Indexing
- Identify fields that are frequently used for filtering or searching and consider adding database indexes to improve query performance.
- Use Django's db_index=True option for fields that are commonly used in queries.
Model Methods and Properties
- Define methods on models to encapsulate business logic or perform complex operations related to the model's data.
- Use properties to compute derived values or provide convenient access to related data.
Meta Options
- Utilize Django's Meta class to provide additional options and metadata for models.
- Specify the ordering of query results using the ordering attribute.
- Define unique constraints using unique_together to enforce unique combinations of fields.
Use Migrations
- Make use of Django's migration system to manage changes to the database schema over time.
- Create and apply migrations whenever there are changes to the models or the database schema.
Testing
- Write unit tests for models to ensure their behavior and interactions are correct.
- Test model methods, properties, and relationships to verify their functionality.
Code Organization
- Organize models into separate modules or files based on related functionality or domain.
- Consider using Django's app structure to group models logically.
Documentation
- Add comments and docstrings to models, fields, and methods to provide clarity and make the code easier to understand.
- Document any assumptions, constraints, or special considerations related to the model's design or usage.
Performance Optimization
- Use Django's select_related and prefetch_related methods to minimize the number of database queries and optimize related data retrieval.
- Be mindful of database hits when working with large datasets and consider using pagination or other strategies to limit query results.
Model should be BIG
- Model class should be big, complex so that its methods can be called in different places, improving clean code and reusable code
Tips
- Use cached_property when you want to save computing resources
- Use Django Signal or overriding save if possible to improve decoupled applications

8. Advance Topics

8.1 Signal

Django Signal allows different components of a Django application to send and receive notifications about specific events, enables loose coupling and promote modularity by facilitating communication between application parts without direct dependencies. This powerful mechanism can be used to extend functionality, integrate with third-party apps, and perform asynchronous tasks, making Django Signals a valuable tool for developers.

However, this also means improving the complexity of your application, be aware of that.

Pros

Loose coupling: The Signal enables modular and maintainable code by decoupling application components.
Extensibility: Custom signals and receivers facilitate easy addition of new functionality.
Asynchronous execution: This feature supports background handling for improved performance.
Integration with third-party apps: This feature allows seamless integration with external components.

Cons

Complexity: Signals can make codebase harder to understand and track interactions.
Lack of explicitness: Communication between components is not explicit in code.
Potential performance impact: Handling a large number of signals and receivers may introduce overhead.
Debugging and testing challenges: Debugging and testing can be more complex compared to direct method calls.

8.2 Overriding Save Method

Django's "Override Save Method" refers to the ability to customize the default behavior of saving data in a Django model. By overriding the save() method, developers can define custom logic that executes before or after saving an object to the database. This feature allows for data validation, pre-processing, or post-processing tasks to be performed during the save operation. It provides flexibility and control over how data is stored and manipulated in Django models, making it a powerful tool for tailoring the saving behavior to specific application requirements.

Example

class Author(models.Model):

    def save(self, *args, **kwargs):

        assert self.price > 0

        self.is_expensive = False

        if self.price > 100:

            self.is_expensive = True

        return super().save(*args, **kwargs)

Pros

Customization: Overriding the save() method allows for custom logic during the saving process, enabling tailored data handling and manipulation.
Flexibility: Developers have full control over how data is saved in Django models, allowing for specific requirements to be met.
Data validation: The save() method can be used to perform data validation before saving, ensuring data integrity.
Pre-processing and post-processing: By overriding save(), developers can perform additional tasks before or after the saving operation, such as data transformations or updates.

Cons

Increased complexity: Overriding the save() method can introduce complexity to the codebase, especially when custom logic becomes intricate.
Maintenance challenges: Custom save() methods may require careful maintenance and documentation to ensure consistency and avoid conflicts.
Potential errors: Inaccurate implementation of save() overrides can lead to unintended consequences or data corruption if not handled properly.
Testing difficulties: Testing save() overrides can be more involved compared to testing standard save operations, requiring comprehensive test coverage.

Both overriding save method and signal have pros and cons, however, we should not mix them together if not really needed.

8.3 Django cached_property

Django's cached_property is a decorator provided by the Django web framework that allows the caching of the result of a property method. It is used to efficiently store and retrieve calculated or expensive property values in Django models or other Python classes. While cached_property offers advantages in terms of performance and code optimization, it also has certain considerations to keep in mind.

from django.utils.functional import cached_property
from django.db.models import Q

class Book(models.Model):

    @cached_property

    def related_books(self):

        return Book.objects.filter(Q(category=self.category) | Q(author=self.author))

Pros

Improved performance: Caching property values boosts execution speed.
Reduced resource usage: Saves computational resources by avoiding redundant computations or queries.
Simplified code: Provides a clean and concise caching mechanism.
Compatibility with Django ORM: Easily integrates with Django's ORM system.

Cons

Memory usage: Caching large or numerous values may increase memory consumption.
Stale data risk: Cached values can become outdated if underlying data changes.
Limited applicability: Best suited for stable and expensive-to-compute properties.
Over-caching potential: Inappropriate caching can lead to bloated cache and performance issues.

8.4 Django Manager

Django Managers are a fundamental part of the Django web framework, providing an interface to interact with database models. Managers offer a way to query, create, update, and delete objects in the database. While managers provide numerous benefits, they also have certain considerations that should be taken into account.

class BookManager(models.Manager):

    def comic(self):

        return self.get_queryset().filter(category='comic')



class Book(models.Model):

    author = models.ForeignKey('Author', on_delete=models.CASCADE, related_name='book')

    title = models.CharField(max_length=200)

    

    objects = BookManager()





Book.objects.comic()

When to use

Custom queries: Managers are useful when you need to define custom queries to retrieve specific subsets of data from the database. They allow you to encapsulate complex query logic and reuse it throughout your application.
Data manipulation: Managers can be employed when you want to perform data manipulation operations on multiple objects simultaneously. They provide a convenient way to update, delete, or create objects in bulk.
Custom methods: Managers enable the addition of custom methods to model classes, which can encapsulate frequently performed tasks or provide additional functionality specific to the model.
Encapsulating business logic: Managers can be utilized to encapsulate business logic related to database operations. This promotes clean and modular code organization, separating database-related concerns from other parts of the application.

Pros

Abstraction layer: Managers act as an abstraction layer between the application code and the database, providing a consistent and unified interface for interacting with the data.
Code reusability: Managers allow you to encapsulate commonly used queries or operations, promoting code reuse and reducing duplication across different parts of the application.
Query optimization: Managers provide opportunities for query optimization by allowing the use of select_related(), prefetch_related(), and other query optimization techniques provided by Django's ORM.
Separation of concerns: Managers enable the separation of database-related logic from other application logic, resulting in a more modular and maintainable codebase.

Cons

Learning curve: Managers require familiarity with Django's ORM and its query API, which may have a learning curve for developers who are new to Django.
Overuse potential: There is a risk of overusing managers and putting too much logic in them, resulting in tightly coupled code and decreased maintainability.
Increased complexity: Managers can introduce additional complexity to the codebase, especially when dealing with complex queries or custom methods.
Limited control: Managers operate at the model level, which means they may not provide fine-grained control over specific database operations, such as low-level query optimizations or advanced transaction handling.

8.5 Django Queryset

Django Queryset customization refers to the ability to modify and extend Django's Queryset API to tailor database queries according to specific requirements. Queryset customization offers flexibility in querying and retrieving data from the database, but it also comes with considerations that should be taken into account.

class BookQueryset(models.QuerySet):

    def kids(self):

        return self.filter(applicable_age__lte=10)



    def american_author(self):

        return self.filter(author__country="USA")





class BookManager(models.Manager):

def get_queryset(self):

return BookQuerySet(self.model, using=self._db)



    def comic(self):

        return self.get_queryset().filter(category='comic')





class Book(models.Model):

    author = models.ForeignKey('Author', on_delete=models.CASCADE, related_name='book')

    title = models.CharField(max_length=200)

    applicable_age = models.IntegerField(default=18)

    objects = BookManager()





Book.objects.comic().american_author().kids()

Book.objects.get_queryset().american_author().kids()

When to use

Complex queries: Queryset customization is beneficial when dealing with complex database queries that cannot be easily achieved using Django's built-in query methods. It allows you to construct custom queries using low-level SQL or advanced filtering techniques.
Performance optimization: Customizing Querysets can be valuable for optimizing query performance. You can leverage Queryset customization to fine-tune queries, apply selective optimizations, or use database-specific features to improve query execution times.
Data transformation and aggregation: When you need to perform data transformation, aggregation, or other advanced data manipulations, Queryset customization allows you to annotate, aggregate, or perform custom calculations on query results.
Integration with external systems: Queryset customization can be useful when integrating Django with external systems or third-party libraries that require specific query customization or extensions.

Pros

Flexibility: Queryset customization provides flexibility in constructing complex and tailored queries, enabling fine-grained control over data retrieval and manipulation.
Performance optimization: Customizing queries allows for optimization techniques, such as query hinting, indexing, or utilizing database-specific features, to enhance query performance and efficiency.
Data manipulation and aggregation: Queryset customization allows for data transformation, aggregation, and calculations directly within the query, reducing the need for additional post-processing or data manipulation in application code.
Integration with external systems: Queryset customization enables seamless integration with external systems or libraries by accommodating specific query requirements.

Cons

Increased complexity: Customizing Querysets can introduce complexity, especially when dealing with intricate or low-level SQL queries. This complexity may require a deeper understanding of database systems and query optimization techniques.
Portability concerns: Queryset customization that relies heavily on database-specific features or syntax may limit the portability of the application to other database backends.
Maintenance challenges: Complex query customizations may require thorough testing, documentation, and careful maintenance to ensure correctness, performance, and compatibility as the application evolves.
Learning curve: Customizing Querysets may require familiarity with Django's Query API, database-specific query syntax, and performance optimization techniques, which can involve a learning curve for developers who are new to these concepts.

You can find another advance blog post about Manager and Queryset from us here

9. Conclusion

In this article, we explored the concepts of Django models and databases. We learned how to create models, work with fields and relationships, perform database migrations, and optimize database queries. By following best practices, you can build robust and scalable Django applications with efficient data storage and retrieval. Keep exploring Django's rich feature set to enhance your web development skills.

FAQs

Q1: Can I change the field type of a model after applying migrations?

Yes, you can change the field type of a model after applying migrations. However, you need to create a new migration to reflect the changes and apply it to the database.

Q2: How can I add a custom method to a Django model?

You can add custom methods to a Django model by defining the method within the model class. These methods can then be called on instances of the model.

Q3: Can I use multiple databases with Django models?

Yes, Django supports working with multiple databases. You can configure and specify different database connections for different models or even for different parts of your application.

Tips: Multiple databases are not recommended due to complexity and issues

Q4: What is the purpose of Django's related_name attribute?

The related_name attribute is used to specify the reverse name for a relationship. It allows you to access related objects in a more intuitive way, especially in cases where multiple relationships are defined between models.

Q5: How can I handle data migrations when deploying my Django application?

When deploying a Django application, you can use tools like Django's migrate command or third-party migration management tools to handle data migrations. These tools help ensure that your database schema is in sync with your models while preserving existing data.

Q6: Are Django models compatible with different database engines?

Yes, Django models are compatible with different database engines. Django's ORM provides an abstraction layer that handles the differences between database engines, allowing you to write database-agnostic code.

Tips: PostgreSQL is the recommended database for Django

Q7: Can I create indexes on specific fields in Django models?

Yes, you can create indexes on specific fields in Django models to improve query performance. Django provides the db_index option for fields to specify whether an index should be created for the field.

Tips: Only create indexes for those columns that are usually queried.