Skip to content
All projects
Tools & UtilitiesOpen source

Anime Genre Analyst & Reporting Engine

An automated, vectorized ETL pipeline that processes denormalized datasets to analyze genre correlations, compute sub-genre synergies, and generate comprehensive analytical reports.

Source code

Tech stack

Languages
Frameworks & libraries

Performance & metrics

  • Eliminated native Python iteration overhead by enforcing C-level vectorized operations (apply, groupby, agg) for large-scale DataFrame mutations
  • Prevented Matplotlib memory leaks by explicitly managing the visualization lifecycle and enforcing immediate garbage collection of inactive Figure canvases

Why I built it

Processing raw, denormalized datasets—such as comma-separated arrays packed into single columns—is error-prone and inefficient. I engineered this pipeline to safely ingest malformed CSVs, dynamically resolve nested string relationships, and autonomously generate statistical reports, transforming raw data into clear insights without manual intervention.

Architecture & decisions

Core data manipulation is offloaded to Pandas' C-backed vectorized operations. Complex 1:N relationships are flattened via the explode API and processed through optimized groupby aggregations. To guarantee fault tolerance, the ingestion layer utilizes a sequential multi-encoding fallback (UTF-8, Latin-1, CP1252) preventing runtime crashes on corrupted files. The visualization layer renders entirely in-memory via Matplotlib, serializing outputs into deterministic, timestamped artifact directories (Markdown, PNG, TXT).

Key features

  1. 01Fault-tolerant CSV ingestion routine with automatic multi-encoding fallback
  2. 02Vectorized transformation of nested comma-separated strings into 1:N relational rows
  3. 03Statistical aggregation engine computing primary and sub-genre performance metrics
  4. 04Automated serialization of timestamp-isolated directories containing Markdown reports and charts