Fix N+1 queries in sessão/relatórios and remove unused painel views

CLAUDE.md

@@ -0,0 +1,145 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+SAPL (Sistema de Apoio ao Processo Legislativo) is a Django-based legislative management system used by Brazilian municipal and state legislative houses. It manages bills, parliamentary sessions, committees, norms, protocols, and related legislative workflows.
+
+## Commands
+
+### Development
+
+```bash
+# Run dev server
+python manage.py runserver
+
+# Docker (dev, without bundled DB)
+docker-compose -f docker/docker-compose-dev.yml up
+
+# Docker (dev, with PostgreSQL container)
+docker-compose -f docker/docker-compose-dev-db.yml up
+```
+
+### Database Setup (local PostgreSQL)
+
+```bash
+sudo -u postgres psql -c "CREATE ROLE sapl LOGIN ENCRYPTED PASSWORD 'sapl' NOSUPERUSER INHERIT CREATEDB NOCREATEROLE NOREPLICATION;"
+sudo -u postgres psql -c "CREATE DATABASE sapl WITH OWNER=sapl ENCODING='UTF8' LC_COLLATE='pt_BR.UTF-8' LC_CTYPE='pt_BR.UTF-8' CONNECTION LIMIT=-1 TEMPLATE template0;"
+python manage.py migrate
+```
+
+### Testing
+
+```bash
+# All tests (reuses DB by default for speed)
+pytest
+
+# Single test file or test function
+pytest sapl/materia/tests/test_materia.py
+pytest sapl/materia/tests/test_materia.py::test_function_name
+
+# Force DB recreation
+pytest --create-db
+
+# With coverage
+pytest --cov=sapl
+```
+
+Tests require `DJANGO_SETTINGS_MODULE=sapl.settings` (set in `pytest.ini`). All tests must be marked with `@pytest.mark.django_db`. The `conftest.py` root fixture provides an `app` fixture (WebTest `DjangoTestApp`).
+
+### Linting / Formatting
+
+```bash
+flake8 .
+isort .
+autopep8 --in-place <file.py>
+```
+
+### Restore Database from Backup
+
+```bash
+./scripts/restore_db.sh -f /path/to/dump
+./scripts/restore_db.sh -f /path/to/dump -p 5433  # Docker port
+```
+
+## Architecture
+
+### Django Apps
+
+Apps are under `sapl/` and follow domain boundaries:
+
+| App | Domain |
+|-----|--------|
+| `base` | `CasaLegislativa` (legislative house config), `AppConfig`, `Autor` (authorship) |
+| `parliamentary` | `Parlamentar`, `Legislatura`, `SessaoLegislativa`, `Coligacao` |
+| `materia` | Bills (`MateriaLegislativa`), types, tracking, annexes |
+| `norma` | Laws/norms (`NormaJuridica`) and hierarchies |
+| `sessao` | Plenary sessions, agenda, attendance, voting |
+| `comissoes` | Committees (`Comissao`) and meetings (`Reuniao`) |
+| `protocoloadm` | Administrative protocols and document intake |
+| `compilacao` | Structured/articulated texts (LexML-like tree structure) |
+| `lexml` | LexML XML standard integration |
+| `audiencia` | Public hearings |
+| `painel` | Real-time session display panel |
+| `relatorios` | PDF report generation |
+| `api` | REST API entry point (auto-generated ViewSets) |
+| `crud` | Generic CRUD base views |
+| `rules` | Business rules and permission definitions |
+
+### REST API
+
+The API uses a custom `drfautoapi` package (`drfautoapi/drfautoapi.py`) that auto-generates DRF ViewSets, Serializers, and FilterSets from Django models. Authentication is Token + Session. Permissions use a custom `SaplModelPermissions` class that maps HTTP methods to Django model permissions.
+
+OpenAPI 3.0 docs are generated by drf-spectacular.
+
+### Caching
+
+- **Default:** File-based (`/var/tmp/django_cache`)
+- **Production:** Redis via django-redis; configured at startup by `configure_redis_cache()` in `sapl/settings.py`
+- **Cache key prefix:** `cache:{POD_NAMESPACE}:` (namespace-isolated for multi-tenant k8s)
+- **Rate limiter state** is shared via Redis keys
+
+### Feature Flags
+
+django-waffle is used for feature flags. Switches (global on/off) can be toggled via:
+
+```bash
+python manage.py waffle_switch <switch_name> on|off
+```
+
+### Key Environment Variables
+
+| Variable | Purpose |
+|----------|---------|
+| `DATABASE_URL` | PostgreSQL connection string |
+| `SECRET_KEY` | Django secret key |
+| `DEBUG` | Debug mode |
+| `REDIS_URL` | Redis host:port |
+| `CACHE_BACKEND` | `file` or `redis` |
+| `POD_NAMESPACE` | K8s namespace (used in cache key prefix) |
+| `USE_SOLR` | Enable Haystack/Solr full-text search |
+| `SOLR_URL` / `SOLR_COLLECTION` | Solr connection |
+
+### Docker Build
+
+The production build requires a MaxMind GeoLite2-ASN license key (for nginx ASN-based bot blocking):
+
+```bash
+docker build --secret id=maxmind_key,src=.env -f docker/Dockerfile -t sapl:local .
+```
+
+Optional build args: `WITH_NGINX`, `WITH_GRAPHVIZ`, `WITH_POPPLER`, `WITH_PSQL_CLIENT`.
+
+### Key File Locations
+
+| File | Purpose |
+|------|---------|
+| `sapl/settings.py` | All Django settings, including cache/rate-limit setup |
+| `pytest.ini` | Test configuration (DJANGO_SETTINGS_MODULE, addopts) |
+| `conftest.py` | Root pytest fixtures |
+| `drfautoapi/drfautoapi.py` | Auto-API generation logic |
+| `docker/startup_scripts/start.sh` | Container entrypoint (migrations, waffle, gunicorn) |
+| `requirements/requirements.txt` | Production deps |
+| `requirements/test-requirements.txt` | Test deps |
+| `requirements/dev-requirements.txt` | Dev/lint deps |

sapl/materia/migrations/0088_fix_view_materiaemtramitacao.py

@@ -0,0 +1,69 @@
+from django.db import migrations, models
+
+_OLD_VIEW = """
+    create or replace view materia_materiaemtramitacao as
+    select m.id as id,
+           m.id as materia_id,
+           t.id as tramitacao_id,
+           t.unidade_tramitacao_destino_id as unidade_tramitacao_atual_id
+    from materia_materialegislativa m
+    inner join materia_tramitacao t on (m.id = t.materia_id)
+    where t.id = (select max(id) from materia_tramitacao where materia_id = m.id)
+    order by m.id DESC
+"""
+
+_NEW_VIEW = """
+    CREATE OR REPLACE VIEW materia_materiaemtramitacao AS
+    SELECT
+        m.id,
+        m.id AS materia_id,
+        t.id AS tramitacao_id,
+        t.unidade_tramitacao_destino_id AS unidade_tramitacao_atual_id
+    FROM materia_materialegislativa m
+    JOIN LATERAL (
+        SELECT
+            t.id,
+            t.unidade_tramitacao_destino_id
+        FROM materia_tramitacao t
+        WHERE t.materia_id = m.id
+        ORDER BY t.id DESC
+        LIMIT 1
+    ) t ON true;
+"""
+
+
+class Migration(migrations.Migration):
+    # CREATE INDEX CONCURRENTLY cannot run inside a transaction.
+    atomic = False
+
+    dependencies = [
+        ('materia', '0087_update_viewdb_materiaemtramitacao'),
+    ]
+
+    operations = [
+        migrations.RunSQL(sql=_NEW_VIEW, reverse_sql=_OLD_VIEW),
+        migrations.SeparateDatabaseAndState(
+            database_operations=[
+                migrations.RunSQL(
+                    sql="""
+                        CREATE INDEX CONCURRENTLY IF NOT EXISTS
+                            tram_materia_id_desc
+                        ON materia_tramitacao (materia_id, id DESC)
+                    """,
+                    reverse_sql="""
+                        DROP INDEX CONCURRENTLY IF EXISTS
+                            tram_materia_id_desc
+                    """,
+                ),
+            ],
+            state_operations=[
+                migrations.AddIndex(
+                    model_name='tramitacao',
+                    index=models.Index(
+                        fields=['materia', '-id'],
+                        name='tram_materia_id_desc',
+                    ),
+                ),
+            ],
+        ),
+    ]

sapl/materia/models.py

@@ -1350,6 +1350,9 @@ class Meta:
         verbose_name = _('Tramitação')
         verbose_name_plural = _('Tramitações')
         ordering = ('-data_tramitacao', '-id')
+        indexes = [
+            models.Index(fields=['materia', '-id'], name='tram_materia_id_desc'),
+        ]
 
     def __str__(self):
         return _('%(materia)s | %(status)s | %(data)s') % {

sapl/painel/urls.py

@@ -1,8 +1,7 @@
 from django.conf.urls import url
 
 from .apps import AppConfig
-from .views import (cronometro_painel, get_dados_painel, painel_mensagem_view,
-                    painel_parlamentar_view, painel_view, painel_votacao_view,
+from .views import (cronometro_painel, get_dados_painel, painel_view,
                     switch_painel, verifica_painel, votante_view)
 
 app_name = AppConfig.name
@@ -11,12 +10,8 @@
     url(r'^painel-principal/(?P<pk>\d+)$', painel_view,
         name="painel_principal"),
     url(r'^painel/(?P<pk>\d+)/dados$', get_dados_painel, name='dados_painel'),
-    url(r'^painel/mensagem$', painel_mensagem_view, name="painel_mensagem"),
-    url(r'^painel/parlamentar$', painel_parlamentar_view,
-        name='painel_parlamentar'),
     url(r'^painel/switch-painel$', switch_painel,
         name="switch_painel"),
-    url(r'^painel/votacao$', painel_votacao_view, name='painel_votacao'),
     url(r'^painel/verifica-painel$', verifica_painel,
         name="verifica_painel"),
     url(r'^painel/cronometro$', cronometro_painel, name='cronometro_painel'),

sapl/painel/views.py

@@ -327,21 +327,6 @@ def verifica_painel(request):
     return resposta
 
 
-@user_passes_test(check_permission)
-def painel_mensagem_view(request):
-    return render(request, 'painel/mensagem.html')
-
-
-@user_passes_test(check_permission)
-def painel_parlamentar_view(request):
-    return render(request, 'painel/parlamentares.html')
-
-
-@user_passes_test(check_permission)
-def painel_votacao_view(request):
-    return render(request, 'painel/votacao.html')
-
-
 @user_passes_test(check_permission)
 def cronometro_painel(request):
     request.session[request.GET['tipo']] = request.GET['action']

sapl/relatorios/forms.py

@@ -543,9 +543,7 @@ class RelatorioMateriasTramitacaoFilterSet(django_filters.FilterSet):
     @property
     def qs(self):
         parent = super(RelatorioMateriasTramitacaoFilterSet, self).qs
-        return parent.distinct().order_by(
-            '-materia__ano', 'materia__tipo', '-materia__numero'
-        )
+        return parent.order_by('-materia__ano', 'materia__tipo', '-materia__numero')
 
     class Meta:
         model = MateriaEmTramitacao

sapl/relatorios/templates/pdf_sessao_plenaria_gerar.py

@@ -7,6 +7,8 @@
 import os
 import time
 import logging
+from xml.sax.saxutils import escape
+
 from django.template.defaultfilters import safe
 from django.utils.html import strip_tags
 from trml2pdf import parseString
@@ -198,6 +200,33 @@ def presenca(lst_presenca_sessao, lst_ausencia_sessao):
     return tmp
 
 
+def correspondencias(lst_correspondencias):
+    tmp = ''
+    if lst_correspondencias:
+        tmp += '\t\t<para style="P1">Correspondências</para>\n'
+        tmp += '\t\t<para style="P2">\n'
+        tmp += '\t\t\t<font color="white"> <br/></font>\n'
+        tmp += '\t\t</para>\n'
+        tmp += '<blockTable style="repeater" repeatRows="1" colWidths="3cm,4cm,3.5cm,6.5cm">\n'
+        tmp += '<tr><td>Tipo</td><td>Documento</td><td>Interessado</td><td>Assunto</td></tr>\n'
+        for c in lst_correspondencias:
+            tmp += '<tr>'
+            tmp += '<td><para style="P4">' + \
+                escape(str(c['tipo'])) + '</para></td>\n'
+            tmp += '<td><para style="P4">' + escape(str(c['epigrafe'])) + \
+                ' - ' + escape(str(c['data'])) + '</para></td>\n'
+            tmp += '<td><para style="P4">' + \
+                escape(str(c['interessado'] or '')) + '</para></td>\n'
+            tmp += '<td><para style="P4">' + \
+                escape(str(c['assunto'] or '')) + '</para></td>\n'
+            tmp += '</tr>\n'
+        tmp += '</blockTable>\n'
+        tmp += '\t\t<para style="P2">\n'
+        tmp += '\t\t\t<font color="white"> <br/></font>\n'
+        tmp += '\t\t</para>\n'
+    return tmp
+
+
 def expedientes(lst_expedientes):
     tmp = ''
     if lst_expedientes:
@@ -415,7 +444,7 @@ def consideracoes(lst_consideracoes):
     return tmp
 
 
-def principal(rodape_dic, imagem, inf_basicas_dic, cont_mult_dic, lst_mesa, lst_presenca_sessao, lst_ausencia_sessao, lst_expedientes, lst_expediente_materia, lst_expediente_materia_vot_nom, lst_oradores_expediente, lst_presenca_ordem_dia, lst_votacao, lst_votacao_vot_nom, lst_oradores_ordemdia, lst_oradores, lst_ocorrencias, lst_consideracoes):
+def principal(rodape_dic, imagem, inf_basicas_dic, cont_mult_dic, lst_mesa, lst_presenca_sessao, lst_ausencia_sessao, lst_correspondencias, lst_expedientes, lst_expediente_materia, lst_expediente_materia_vot_nom, lst_oradores_expediente, lst_presenca_ordem_dia, lst_votacao, lst_votacao_vot_nom, lst_oradores_ordemdia, lst_oradores, lst_ocorrencias, lst_consideracoes):
     """
     """
     arquivoPdf = str(int(time.time() * 100)) + ".pdf"
@@ -440,6 +469,7 @@ def principal(rodape_dic, imagem, inf_basicas_dic, cont_mult_dic, lst_mesa, lst_
     ordenacao = ResumoOrdenacao.objects.first()
     dict_ord_template = {
         'cont_mult': multimidia(cont_mult_dic),
+        'correspondencia': correspondencias(lst_correspondencias),
         'exp': expedientes(lst_expedientes),
         'id_basica': inf_basicas(inf_basicas_dic),
         'lista_p': presenca(lst_presenca_sessao, lst_ausencia_sessao),
@@ -473,13 +503,15 @@ def principal(rodape_dic, imagem, inf_basicas_dic, cont_mult_dic, lst_mesa, lst_
             tmp += dict_ord_template[ordenacao.decimo_terceiro]
             tmp += dict_ord_template[ordenacao.decimo_quarto]
             tmp += dict_ord_template[ordenacao.decimo_quinto]
+            tmp += dict_ord_template[ordenacao.decimo_sexto]
         except KeyError as e:
             logger.error("KeyError: " + str(e) + ". Erro ao tentar utilizar "
                               "configuração de ordenação. Utilizando ordenação padrão.")
             tmp += inf_basicas(inf_basicas_dic)
             tmp += multimidia(cont_mult_dic)
             tmp += mesa(lst_mesa)
             tmp += presenca(lst_presenca_sessao, lst_ausencia_sessao)
+            tmp += correspondencias(lst_correspondencias)
             tmp += expedientes(lst_expedientes)
             tmp += expediente_materia(lst_expediente_materia)
             tmp += expediente_materia_vot_nom(lst_expediente_materia_vot_nom)
@@ -497,6 +529,7 @@ def principal(rodape_dic, imagem, inf_basicas_dic, cont_mult_dic, lst_mesa, lst_
         tmp += multimidia(cont_mult_dic)
         tmp += mesa(lst_mesa)
         tmp += presenca(lst_presenca_sessao, lst_ausencia_sessao)
+        tmp += correspondencias(lst_correspondencias)
         tmp += expedientes(lst_expedientes)
         tmp += expediente_materia(lst_expediente_materia)
         tmp += expediente_materia_vot_nom(lst_expediente_materia_vot_nom)