Skip to content

Refactor: Codebase cleanup, migration optimizations, and API enhancements - #62

Merged
Genarito merged 14 commits into
devfrom
code-clean
Sep 25, 2026
Merged

Genarito merged 14 commits into
devfrom
code-clean

Conversation

@mauri101-Ar

Copy link
Copy Markdown
Collaborator

This PR introduces a significant cleanup of the Modulector codebase, optimizes database migrations, and improves API responses.
Key Changes:

  • Codebase Cleanup: Removed obsolete models (OldRefSeqMapping, GeneSymbolMapping), outdated mapper scripts, and the entire processors directory. Replaced the deprecated mapping functionality with the GeneAliases model.
  • Migrations Optimization: Squashed 47 old migrations into a clean history (0001_initial and 0002_load_data). Added tqdm progress bars, file existence validation, and bulk_create processing, which significantly reduces RAM usage and execution times for large datasets like mirDIP, miRTarBase, and EPIC.
  • Data Loading Refactor: Moved the data processing logic for HGNC and SM2miR datasets directly into native Django migrations. Updated DEPLOYING.md with instructions for downloading the required source files manually.
  • API Enhancements: Updated the /mirna-target-validation/ endpoint to support inclusive alias searches for both miRNAs and genes, exposing these aliases in the JSON response.
  • API Enhancements: Added the mirna field explicitly to the JSON responses for the /diseases/ and /drugs/ endpoints to improve result clarity.
  • Bug Fixes & Maintenance: Fixed the failing EPIC test by handling the "chr" prefix issue to match the database format. Added the mirna_mature.txt file directly to the repository to ensure availability.

…Gencode to gencode across views, tests, and documentation
… bulk inserts, and add tqdm progress bars to all migration tasks
…lementing bulk processing for EPIC and miRTarBase datasets
…light checks, unified logging, and optimized bulk creation processes.
@Genarito
Genarito changed the base branch from main to dev September 19, 2026 18:55
Comment thread .gitignore Outdated
*.sql.gz
modulector/files/tmp_db.csv
docker-compose.mauri_dev.yml
code-clean.md

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sacar este y el otro archivo, no son cosas que vamos a tener todos y los LLMs generan un millón de esos, así que podríamos llenar el .gitignore de este tipo de archivos, directamente no poner

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hecho!

Comment thread docker-compose.dev.yml Outdated
image: postgres:16
container_name: modulector_dev_db
restart: always
restart: no

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Poner restart: always para ser consistente con todo lo demás

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hecho!

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
25.5% Duplication on New Code (required ≤ 3%)
B Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

from django.utils import timezone
source_mirdip = MirnaSource.objects.create(
name='mirdip',
site_url='http://ophid.utoronto.ca/mirDIP/',
from django.utils import timezone
source_mirdip = MirnaSource.objects.create(
name='mirdip',
site_url='http://ophid.utoronto.ca/mirDIP/',
@Genarito
Genarito merged commit 60c748d into dev Sep 25, 2026
1 of 2 checks passed
@Genarito
Genarito deleted the code-clean branch September 25, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants