We're going to build a simple app that uses advanced PostgreSQL full-text search features, with results that update as the user enters their query.
This includes:
- using HTMX to fetch results as the user types (without extra JavaScript)
- allowing the user to enter search engine style queries (i.e. "python or django")
- adding a Django management command to quickly insert test data
Here's a video of the working product:
Previewing the project
Want to see a live version of the app? You can create a copy of this project now and try it out.
Copy the FullTextSearchApp Project
Setting up the Django app
Install packages and create the Django application.
pip install --upgrade django faker
django-admin startproject eventsearch .
python3 manage.py startapp core
Add core to the INSTALLED_APPS list.
# settings.py
INSTALLED_APPS = [
"core",
...
]
Adding the templates
- Create a directory named
templateswithin thecoreapp. - Create a file named
search.htmlwithin thetemplatesdirectory.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<script src="https://unpkg.com/htmx.org"></script>
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/css/bootstrap.min.css"
rel="stylesheet"
integrity="sha384-EVSTQN3/azprG1Anm3QDgpJLIm9Nao0Yz1ztcQTwFspd3yD65VohhpuuCOmLASjC"
crossorigin="anonymous">
<link href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.10.0/font/bootstrap-icons.css"
rel="stylesheet">
<style>
html,
body {
height: 100%;
}
#content {
display: flex;
flex-direction: column;
height: 100%;
}
#events-content {
background-color: rgb(246, 247, 248);
flex: 1;
}
#events-content-inner {
max-width: 600px;
width: 100%;
}
#event-search-input {
border-top-left-radius: 8px;
border-bottom-left-radius: 8px;
border: 1px solid rgb(217, 217, 217);
border-right: none;
max-width: 600px;
outline: none;
}
#search-submit-btn {
background-color: rgb(246, 88, 88);
border-top-right-radius: 8px;
border-bottom-right-radius: 8px;
border: none;
color: #fff;
width: 42px;
}
.search-result {
border-bottom: 1px solid #ccc;
}
.search-result-inner {
display: flex;
gap: 1rem;
}
.event-thumbnail {
border-radius: 8px;
width: 160px;
height: 100px;
}
.event-description {
font-size: 13px;
}
.event-date {
color: rgb(124, 111, 80);
font-weight: 500;
}
</style>
</head>
<body>
<div id="content">
<form method="post">
{% csrf_token %}
<div id="event-search" class="d-flex justify-content-center p-3">
<div class="d-flex justify-content-center w-50">
<input id="event-search-input"
name="q"
class="w-100 p-2"
type="text"
value="{{ query }}"
placeholder="Search events"
hx-post="/"
hx-select="#events-content-inner"
hx-swap="outerHTML"
hx-trigger="input changed delay:500ms, search"
hx-target="#events-content-inner" />
<button id="search-submit-btn" type="submit">
<i class="bi bi-search"></i>
</button>
</div>
</div>
</form>
<div id="events-content" class="d-flex justify-content-center">
<div id="events-content-inner" class="p-4">
{% for event in events %}
<div class="search-result py-4">
<div class="search-result-inner">
<div>
{% if event.image_url %}<img class="event-thumbnail" src="{{ event.image_url }}" />{% endif %}
</div>
<div>
<div class="event-date">{{ event.event_date|date:"l, F d" }} {{ event.event_time }}</div>
<div>{{ event.name }}</div>
<div class="event-description mt-2">{{ event.description }}</div>
</div>
</div>
</div>
{% endfor %}
</div>
</div>
</div>
</body>
</html>
Adding the views
Copy and paste the following into views.py within the core directory.
from django.contrib.postgres.search import SearchQuery
from django.shortcuts import render
from core.models import Event
def search(request):
events, q = [], ""
if request.method == "POST":
q = request.POST.get("q")
if q:
events = Event.objects.filter(
description_vector=SearchQuery(q, search_type="websearch")
)
if not events:
events = Event.objects.all()
return render(request, "core/search.html", {"query": q, "events": events})
Updating URLs
Create urls.py in the core directory.
from django.urls import path
from .views import search
urlpatterns = [
path("", search, name="search")
]
Update the existing urls.py within the project eventsearch directory.
from django.contrib import admin
from django.urls import include, path
urlpatterns = [
path("admin/", admin.site.urls),
path("", include("core.urls")),
]
Adding the database models
Overwrite the existing models.py with the following:
from psycopg2.extensions import register_adapter, AsIs
from django.db import models
from django.db.models import F
from django.contrib.auth.models import User
from django.contrib.postgres.search import SearchVectorField
from django.contrib.postgres.indexes import GinIndex
class PostgresDefaultValueType:
pass
# Wrap the built-in SearchVectorField so that we pass DEFAULT when value is None.
# Otherwise, the Django ORM will try to update the `description_vector` field, which
# PostgreSQL won't allow because it is a GENERATED column.
# Credit for this approach belongs here:
# https://code.djangoproject.com/ticket/21454#comment:28
class DelegatedSearchVectorField(SearchVectorField):
def get_prep_value(self, value):
return value or PostgresDefaultValueType()
register_adapter(PostgresDefaultValueType, lambda _: AsIs('DEFAULT'))
class Event(models.Model):
name = models.CharField(max_length=100)
description = models.TextField()
description_vector = DelegatedSearchVectorField()
image_url = models.URLField(max_length=1024, blank=True, null=True)
event_date = models.DateField()
event_time = models.TimeField()
Adding a Manual Migration for the Search Vector Field
Run the migration command to generate a new empty migration.
python3 manage.py makemigrations core --empty
Populate the empty migration with the following.
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('core', '0001_initial'),
]
operations = [
migrations.RunSQL(
"""
ALTER TABLE core_event ADD COLUMN description_vector tsvector GENERATED ALWAYS AS (
setweight(to_tsvector('english', coalesce(name, '')), 'A') ||
setweight(to_tsvector('english', coalesce(description, '')), 'B')
) STORED;
CREATE INDEX IF NOT EXISTS description_vector_gin ON core_event USING GIN(description_vector);
""",
reverse_sql=
"""
ALTER TABLE core_event DROP COLUMN description_vector;
""",
),
]
The migration will create an automatically updated tsvector by using a GENERATED column. Django 5.0 supports the GeneratedField type, but the SQL expression is too complex to handle that way, so we'll use a manual migration to handle it without the ORM.
Adding a Django Management Command for Test Data
- Create the directory structure
management/commandswithin thecorefolder. - Open a file named
populate_events.pywithin the new directory and enter the following.
from datetime import datetime, timedelta
import random
from django.core.management.base import BaseCommand
from django.contrib.auth.models import User
from django.utils.timezone import make_aware
from faker import Faker
from core.models import Event
class Command(BaseCommand):
def handle(self, *args, **options):
self.stdout.write(self.style.SUCCESS('Starting database population...'))
faker = Faker()
for i in range(10):
first_name = faker.first_name()
last_name = faker.last_name()
User.objects.create_user(username=f'user{i}',
email=f'user{i}@example.com',
password='password',
first_name=first_name,
last_name=last_name)
event_names = [
'Django for Beginners',
'Python Data Science Meetup',
'React Native Workshop',
'Machine Learning Basics',
'Introduction to IoT',
'Blockchain 101',
'Advanced Django Techniques',
'Full-stack Development with Python and JavaScript',
'Mobile App Development Trends',
'Cybersecurity Fundamentals',
]
descriptions = [
'Learn Django through practical examples, from setting up your environment to building and deploying a web application.',
'Explore data science techniques using Python, including statistical analysis, machine learning algorithms, and data visualization.',
'Build your first React Native app by learning the basics of this cross-platform framework and creating a simple application.',
'An introduction to machine learning concepts, covering foundational principles, algorithms, and real-world applications.',
'Getting started with the Internet of Things (IoT): Learn how to connect devices and collect data for analysis and insights.',
'Understanding blockchain technology, from the basics of decentralized networks to smart contracts and cryptocurrency.',
'Deep dive into Django features, exploring advanced functionalities that can help in building robust and scalable web applications.',
'Learn full-stack development from front to back, covering everything from basic HTML/CSS to backend programming and databases.',
'What\'s new in mobile app development: Explore the latest trends, tools, and technologies shaping the future of mobile applications.',
'Protect your digital assets with cybersecurity basics, learning about threats, safeguards, and best practices for online security.',
]
image_urls = [
'https://circumeo.io/static/images/server-racks.png',
'https://circumeo.io/static/images/typewriter.jpg',
'https://circumeo.io/static/images/library.jpg',
'https://circumeo.io/static/images/landscape.jpg',
'https://circumeo.io/static/images/laptop.jpg',
'https://circumeo.io/static/images/latte.jpg',
]
for name, description in zip(event_names, descriptions):
image_url = random.choice(image_urls)
event_date = datetime.now() + timedelta(days=random.randint(1, 90))
event_time = datetime.now().time()
Event.objects.create(name=name, description=description, image_url=image_url, event_date=make_aware(event_date), event_time=event_time)
self.stdout.write(self.style.SUCCESS('Database population completed successfully!'))
Open a shell session in order to run the Django management command.
python3 manage.py populate_events
Up and Running with Full Text Search
That's it! You now have an app up and running that incorporates powerful full-text search capabilities. Not to mention, the search is dynamic as well, without the need for additional JavaScript, thanks to HTMX.