In this article, we will explain how to integrate Django with WebPack and how to manage all JavaScript packages through Yarn. This way of operating will provide us with a series of advantages over traditional development since WebPack will allow us to reuse code between JavaScript modules, an intelligent caching system, and on-demand loading of parts of the JavaScript code.

Before starting, it is recommended to read the previous articles on Django since they are the previous steps to this article:

• Django: Venv under FreeBSD
• Django: MVT, Apps and URLs
• Django: Database Models
• Django: Administration Interface
• Django: DTL(Django Template Language)
• Django: Debug Toolbar
• Django: User Registration and Authentication

The integration of Django with WebPack involves several parts:

• WebPack: Content packager.
• Yarn: JavaScript package manager, boasts of being faster, safer, and more reliable than NPM.
• Webpack-bundle-tracker: WebPack plugin responsible for generating the webpack-stats.json file where all the necessary information is indicated so that the django-webpack-loader package can make the necessary substitutions in the Django templates for the names of the bundle files.
• Django-webpack-loader: Package that will read the webpack-stats.json file and make the necessary substitutions in the templates for the names of the bundle files.

WebPack is a web content packager that packages both assets and JavaScript code. The final output is a single JS file, so by importing this file, we can import style sheets and JS code.

It also allows for “code splitting” and “lazy loading”, so the parts of the content that are requested at that moment in response to certain user actions will be loaded.

On the other hand, Webpack is very useful for avoiding content caching problems. The files are generated with a unique hash as the file name. Therefore, when changes are made to the website, these hashes are regenerated, which are different from the previous hashes. Thus, a client who reloads the website will be requesting a different file, and the browser cache will be discarded, showing the updated content.

To use assets in Django templates, they must be referenced by their name. The problem is that WebPack generates hashed files as described above. To be able to use the files generated by WebPack transparently, we will use the webpack-bundle-tracker plugin and the django-webpack-loader package, which will make the necessary substitutions in our templates with the names of the files from the latest generated bundle. webpack-bundle-tracker generates all the information that django-webpack-loader needs in a file called webpack-stats.json.

Before we start, we must take into account certain aspects of WebPack:

• To generate the dependency tree of the application, Webpack needs a starting point, a JS file that it will read to build the dependency tree. This JS file is called the entry point.
• By defining several entry points, we will be able to split our code into smaller parts. Depending on the part of the website we are visiting, we will load different bundles, making navigation more efficient.
• Loaders are third-party extensions that help Webpack deal with various types of files such as CSS, images, and text files, among others. The goal of loaders is to transform files into modules. Once transformed, Webpack can use them as project dependencies.
• Webpack can also use what it calls Plugins. These are extensions that modify the behavior of Webpack, for example, to perform certain actions depending on the prod/dev environment.
• There are two modes of operation: development and production. The main difference between the two modes is that in production, JS code is minified and optimized.

Yarn is a new JavaScript package manager that is compatible with NPM but operates faster, more securely, and more reliably.

Install NPM (root):

Install yarn globally (root):

JS package managers require a package.json configuration file to function. This file indicates the available commands, versions, and other information associated with the packages. We generate it using the following command:

yarn init v1.22.10  
warning The yes flag has been set. This will automatically answer yes to all questions, which may have security implications.  
success Saved package.json  
Done in 0.02s.

The -y parameter indicates that we want to generate the package.json file with the default parameters.

It will have created the file with the project name rxWodProject. If we use VSCode as our code editor, this will emit a warning. To avoid it, we change it to lowercase:

  "name": "rxwodproject",

As we have already mentioned, webpack-bundle-tracker is the WebPack plugin responsible for generating the webpack-stats.json file, which indicates all the necessary information for the django-webpack-loader package to make the necessary substitutions in the Django templates for the names of the bundle files.

Install WebPack, webpack-cli, webpack-bundle-tracker, and progress-bar-webpack-plugin:

The latest version of webpack-bundle-tracker generates the webpack-stats.json file with a syntax that django-webpack-loader is unable to understand. When trying to load the data, it displays the following error:

TypeError: string indices must be integers

Fortunately, django-webpack-loader is extremely flexible and allows data to be loaded in different ways and from different sources. We only need to extend the webpack_loader.loader.WebpackLoader class, which allows us to load stats from a text file, a database, or any other data source. In my case, I only slightly modified the class to load the data with the new syntax of the webpack-stats.json file.

from webpack_loader.loader import WebpackLoader

class CustomWebpackLoader(WebpackLoader):
    def filter_chunks(self, chunks):
        chunks = [chunk if isinstance(chunk, dict) else {'name': chunk} for chunk in chunks]
        return super().filter_chunks(chunks)

NOTE: There are forums that recommend downgrading the version of webpack-bundle-tracker, but then we will get deprecation messages:

(node:44658) [DEP_WEBPACK_DEPRECATION_ARRAY_TO_SET] DeprecationWarning: Compilation.chunks was changed from Array to Set (using Array method ‘map’ is deprecated)

I think the best option is to use this patch until the developers fix the bug.

We create the assets directory where our JS source code and files generated by WebPack will reside:

alert("Testing WebPack and Django from AlfaExploit!");

We generate a basic WebPack configuration where we use the webpack-bundle-tracker plugin and indicate a single entry file:

var path = require("path")
var webpack = require('webpack')
var BundleTracker = require('webpack-bundle-tracker')
var ProgressBarPlugin = require('progress-bar-webpack-plugin');

module.exports = {
    mode: "production",
    context: __dirname,
    entry: {
        index: './assets/js/index.js',
    },

    output: {
        path: path.resolve('./assets/bundles/'),
        publicPath: '/static/',
        filename: "[name]-[fullhash].js",
    },

    plugins: [
        new BundleTracker({filename: '../../webpack-stats.json'}),
        new ProgressBarPlugin(),
    ],
}

NOTE: The same configuration works for dev but we must define mode: “development”.

Our project will have the following file/directory structure:

|-- assets  
|-- db.sqlite3  
|-- fixes  
|-- manage.py  
|-- node_modules  
|-- package.json  
|-- requirements.txt  
|-- rxWod  
|-- rxWodProject  
|-- usersAuth  
|-- webpack.dev.config.js  
|-- webpack.prod.config.js  
`-- yarn.lock

Django manages assets through its configuration in the settings.py file, first we import the os library since we will need it and add webpack_loader as a Django app.

import os
...

INSTALLED_APPS = [
    ...
    'webpack_loader',
]

Then we configure the parameters related to WebPack:

• STATIC_URL: URL from which all our static content will hang, for example if we want to access an image we will access http://DOMAIN_NAME/STATIC_URL/file.png
• STATICFILES_DIRS: Directory where the static content generated by WebPack is located, when the collectstatic command is executed, the files in this directory will be read and copied to the path configured in STATIC_ROOT.
• STATIC_ROOT: Directory where all the static content of the project will be copied when the collectstatic command is executed, this content will be available under the URL: http://DOMAIN_NAME/STATIC_URL/, this step is only necessary when we pass our project to production and use some web server like Apache/Nginx, for now we just leave it configured.
• WEBPACK_LOADER -> BUNDLE_DIR_NAME : Directory where to find the WebPack bundles, since STATICFILES_DIRS already points directly to that directory we must leave it empty.
  NOTE: If your webpack config outputs the bundles at the root of your staticfiles dir, then BUNDLE_DIR_NAME should be an empty string ‘’, not ‘/’.
• WEBPACK_LOADER -> LOADER_CLASS: We load our patched class, fixes.webpack.CustomWebpackLoader.
• WEBPACK_LOADER -> STATS_FILE: We indicate the name of the file that will be generated by the webpack-bundle-tracker module.

NOTE: With the server integrated with Django, it is not necessary to execute the collectstatic command since it will serve the bundles directly from the STATICFILES_DIRS directory.

STATIC_URL = '/static/'

STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'assets/bundles/'),
)

STATIC_ROOT = '/usr/local/www/rxWod/static/'

WEBPACK_LOADER = {
    'DEFAULT': {
        'BUNDLE_DIR_NAME': '',
        'LOADER_CLASS': 'fixes.webpack.CustomWebpackLoader',
        'STATS_FILE': os.path.join(BASE_DIR, 'webpack-stats.json'),
    }
}

NOTE: As of 6/08/2021, the webpack-bundle-tracker bug has been patched, so the configuration files would be as follows:

var path = require("path")
var webpack = require('webpack')
var BundleTracker = require('webpack-bundle-tracker')
var ProgressBarPlugin = require('progress-bar-webpack-plugin');

module.exports = {
    mode: "production",
    context: __dirname,
    entry: {
        index: './assets/js/index.js',
    },

    output: {
        path: path.resolve('./assets/bundles/'),
        publicPath: '/static/',
        filename: "[name]-[fullhash].js",
    },

    plugins: [
        new BundleTracker({filename: 'webpack-stats.json'}),
        new ProgressBarPlugin(),
    ],
}

The path of BundleTracker -> filename has changed.

STATIC_URL = '/static/'

STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'assets/bundles/'),
)

STATIC_ROOT = '/usr/local/www/rxWod/static/'

WEBPACK_LOADER = {
    'DEFAULT': {
        'BUNDLE_DIR_NAME': '',
        'STATS_FILE': os.path.join(BASE_DIR, 'webpack-stats.json'),
    }
}

In the WEBPACK_LOADER configuration, it is no longer necessary to indicate a LOADER_CLASS.

It should also be noted that the fixes/webpack.py patch is no longer necessary, so we delete the directory:

We define the commands to compile the WebPack bundles, first we delete files generated by previous compilations and then launch a new compilation, in the case of dev we also launch the integrated Django server:

{
  "name": "rxwodproject",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "webpack": "^5.24.4",
    "webpack-bundle-tracker": "^1.0.0-alpha.1",
    "webpack-cli": "^4.5.0"
  },
  "scripts": {
    "prod-build": "rm -rf assets/bundles/*; yarn webpack --config ./webpack.prod.config.js",
    "dev-build": "rm -rf assets/bundles/*; yarn webpack --config ./webpack.dev.config.js; python manage.py runserver",
    "watch": "rm -rf assets/bundles/*; yarn webpack --watch --config ./webpack.dev.config.js"
  }
}

We install the django-webpack-loader plugin and freeze the dependencies:

We compile the bundles with WebPack:

We can leave WebPack in watch mode so that it is continuously recompiling the bundles as soon as changes are detected in the entry files or their dependencies, but if we change the WebPack configuration we must relaunch the command with watch.

It will have generated a file:

assets/bundles/index-[hash].js

total 6  
drwxr-xr-x  2 kr0m  kr0m     3 Mar 10 22:27 .  
drwxr-xr-x  4 kr0m  kr0m     4 Mar 10 22:08 ..  
-rw-r--r--  1 kr0m  kr0m  1305 Mar 10 22:27 index-b68ca7fe24923105061f.js

WebPack is now working, now we need to use the generated bundles in the Django templates, for this we will use django-webpack-loader , this package will read the information from the webpack-stats.json file to load the bundle files from the Django template.

If we visualize the content of the webpack-stats.json file, we can see the correspondence between the name of the entry point and the generated bundle.

    entry: {
        index: './assets/js/index.js',
    },

{"status":"done","chunks":{"index":["index-b68ca7fe24923105061f.js"]},"publicPath":"/static/","assets":{"index-b68ca7fe24923105061f.js":{"name":"index-b68ca7fe24923105061f.js","publicPath":"/static/index-b68ca7fe24923105061f.js"}}}

When we refer to index in our template, we will be loading the index-b68ca7fe24923105061f.js file.

We modify our template to load our JS code:

{% extends 'rxWod/base.html' %}
{% load render_bundle from webpack_loader %}

{% block base_body %}
        {% if routines %}
            <ul>
            {% for routine in routines %}
                <li><a>{{ routine.date }}</a></li>
            {% endfor %}
            </ul>
        {% else %}
            <p>No routines available.</p>
        {% endif %}

    {% render_bundle 'index' %}
{% endblock %}

We start the server:

We can see the template and the message emitted via JS:

Django will automatically use the latest generated bundle. To check it, we can make a change in the JS code:

alert("Second test: Testing WebPack and Django from AlfaExploit!");

We recompile with the bundles:

We check that the new version of the file has been generated:

total 6  
drwxr-xr-x  2 kr0m  kr0m     3 Mar 10 22:27 .  
drwxr-xr-x  4 kr0m  kr0m     4 Mar 10 22:08 ..  
-rw-r--r--  1 kr0m  kr0m  1316 Mar 10 22:55 index-b4463486d2c11cf7787f.js

We start the server:

We access the website and check that the message has been updated correctly without having to touch the template:

To be able to use CSS in Django templates, we must import them inside our JS code, but to do this, we will have to use WebPack loaders: css-loader and style-loader .

The first one allows JS to load the content of the CSS, and the second one allows it to be displayed in the browser’s DOM.

We edit the webpack.ENV.config.js file and add the module section, where we indicate that when it finds a CSS file, it should use the two loaders:

var path = require("path")
var webpack = require('webpack')
var BundleTracker = require('webpack-bundle-tracker')
var ProgressBarPlugin = require('progress-bar-webpack-plugin');

module.exports = {
    mode: "development",
    context: __dirname,
    entry: {
        index: './assets/js/index.js',
    },

    output: {
        path: path.resolve('./assets/bundles/'),
        publicPath: '/static/',
        filename: "[name]-[fullhash].js",
    },

    plugins: [
        new BundleTracker({filename: '../../webpack-stats.json'}),
        new ProgressBarPlugin(),
    ],

    module:{
        rules:[
            {
                test: /\.css$/i,
                use:['style-loader','css-loader']
            }
       ]
    },
}

We generate the CSS file with the following content:

body {
    background-color: lightblue;
}

We import the CSS from the JS code:

import '../css/index.css';
alert("Third test: Testing WebPack and Django from AlfaExploit!");

Remember that our template looks like this:

{% extends 'rxWod/base.html' %}
{% load render_bundle from webpack_loader %}

{% block base_body %}
        {% if routines %}
            <ul>
            {% for routine in routines %}
                <li><a>{{ routine.date }}</a></li>
            {% endfor %}
            </ul>
        {% else %}
            <p>No routines available.</p>
        {% endif %}

    {% render_bundle 'index' %}
{% endblock %}

We recompile using WebPack:

We start the server:

We access the website and check that we have loaded the JS code and our CSS:

In addition, if we look at the files downloaded from the website, we can see a single JS file, the rest of the files are part of the Django debug toolbar , so we can ignore them:

We can load only the JS/CSS part by indicating it as the second parameter in the render_bundle command of the template, but for this we will have to generate the CSS file using the mini-css-extract-plugin plugin.

var path = require("path")
var webpack = require('webpack')
var BundleTracker = require('webpack-bundle-tracker')
var ProgressBarPlugin = require('progress-bar-webpack-plugin');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');

module.exports = {
    mode: "development",
    context: __dirname,
    entry: {
        index: './assets/js/index.js',
    },

    output: {
        path: path.resolve('./assets/bundles/'),
        publicPath: '/static/',
        filename: "[name]-[fullhash].js",
    },

    plugins: [
        new BundleTracker({filename: '../../webpack-stats.json'}),
        new ProgressBarPlugin(),
        new MiniCssExtractPlugin({
            filename: '[name]-[hash].css',
        }),
    ],

    module:{
        rules:[
            {
                test: /\.css$/i,
                use: [MiniCssExtractPlugin.loader, 'css-loader'],
            }
       ]
    },
}

NOTE: When using MiniCssExtractPlugin, the style-loader loader will no longer be necessary.

We recompile using WebPack:

It has generated the CSS file:

total 6  
drwxr-xr-x  2 kr0m  kr0m     4 Mar 10 22:27 .  
drwxr-xr-x  5 kr0m  kr0m     5 Mar 10 22:02 ..  
-rw-r--r--  1 kr0m  kr0m    43 Mar 10 23:27 index-94788942e470c8727cee.css  
-rw-r--r--  1 kr0m  kr0m  3409 Mar 10 23:27 index-94788942e470c8727cee.js

Let’s do the test:

{% extends 'rxWod/base.html' %}
{% load render_bundle from webpack_loader %}

{% block base_body %}
        {% if routines %}
            <ul>
            {% for routine in routines %}
                <li><a>{{ routine.date }}</a></li>
            {% endfor %}
            </ul>
        {% else %}
            <p>No routines available.</p>
        {% endif %}

    {% render_bundle 'index' 'css' %}
{% endblock %}

As we can see, only the CSS has loaded, the JS code alert window has not appeared, and only the CSS file has been downloaded:

Let’s make it load only the JS code:

{% extends 'rxWod/base.html' %}
{% load render_bundle from webpack_loader %}

{% block base_body %}
        {% if routines %}
            <ul>
            {% for routine in routines %}
                <li><a>{{ routine.date }}</a></li>
            {% endfor %}
            </ul>
        {% else %}
            <p>No routines available.</p>
        {% endif %}

    {% render_bundle 'index' 'js' %}
{% endblock %}

There is no need to recompile since we haven’t changed anything in the JS/CSS.

We access the website and as we can see, only the JS has loaded, without applying the CSS:

We can see that the JS file has been downloaded:

If we leave the CSS extractor enabled and don’t indicate anything in the render_bundle command, both the JS code and the CSS content will be loaded, but two files will be downloaded instead of just one as before enabling the extractor:

{% extends 'rxWod/base.html' %}
{% load render_bundle from webpack_loader %}

{% block base_body %}
        {% if routines %}
            <ul>
            {% for routine in routines %}
                <li><a>{{ routine.date }}</a></li>
            {% endfor %}
            </ul>
        {% else %}
            <p>No routines available.</p>
        {% endif %}

    {% render_bundle 'index' %}
{% endblock %}

Normally, CSS is usually loaded at the beginning of the head section:

<head>
    {% render_bundle 'main' 'css' %}

And the JS code at the end of the body:

    {% render_bundle 'main' 'js' %}
</body>

If we need to use an asset such as an image, we can use the file-loader loader and the webpack_static tag in the templates.

We install the loader:

But there is a problem, and that is that webpack_static must be buggy and does not load hashed image files, it always refers to the original name without the hash part. The only solution I have found is not to hash them by telling the file-loader to save the files with the original name name: ‘[name].[ext]’:

var path = require("path")
var webpack = require('webpack')
var BundleTracker = require('webpack-bundle-tracker')
var ProgressBarPlugin = require('progress-bar-webpack-plugin');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');

module.exports = {
    mode: "development",
    context: __dirname,
    entry: {
        index: './assets/js/index.js',
    },

    output: {
        path: path.resolve('./assets/bundles/'),
        publicPath: '/static/',
        filename: "[name]-[fullhash].js",
    },

    plugins: [
        new BundleTracker({filename: '../../webpack-stats.json'}),
        new ProgressBarPlugin(),
        new MiniCssExtractPlugin({
            filename: '[name]-[hash].css',
        }),
    ],

    module:{
        rules:[
            {
                test: /\.css$/i,
                use: [MiniCssExtractPlugin.loader, 'css-loader'],
            },
            {
                test: /\.(png|jpe?g|gif)$/i,
                loader: 'file-loader',
                options: {
                  name: '[name].[ext]',
                },
            },
       ]
       
    },
}

We create the directory where we will store the images and download an example image:

We import the image from our JS:

import '../css/index.css';
import img from '../images/django-logo-positive.png';

alert("Third test: Testing WebPack and Django from AlfaExploit!");

We compile using WebPack:

Our template will look like this:

{% extends 'rxWod/base.html' %}
{% load render_bundle from webpack_loader %}
{% load webpack_static from webpack_loader %}

{% block base_body %}
        {% if routines %}
            <ul>
            {% for routine in routines %}
                <li><a>{{ routine.date }}</a></li>
            {% endfor %}
            </ul>
        {% else %}
            <p>No routines available.</p>
        {% endif %}

    {% render_bundle 'index' %}
    <img src="{% webpack_static 'django-logo-positive.png' %}"/>
{% endblock %}

We can see how it shows the image:

We can see the loaded elements:

The only drawback is that by not generating the hashed file, we lose the ability to discard browser image caching, but for JS/CSS it will continue to work.