Akshay Soner

Backend / Systems Engineer

Backend / Systems Engineer focused on scalable APIs, data-intensive systems, AI integrations, automation, and reliable production deployments.

Run trace of a batch pipeline that resumes from its last completed batch, and the recovery patterns behind four production systems

Large-Scale Data Processing

25M+ records · ~4GB · single pass

  1. Raw Data Source

    not loaded wholesale

  2. Batch Loader

    bounded batches, flat memory

  3. Validation / Dedup

    dedup before writes

  4. Database Write

    locks and round-trips minimized

  5. Monitoring

    progress tracked

The dashed recovery path runs from Database Write back to Batch Loader, skipping Raw Data Source: on failure, the run resumes from the last completed batch — it does not restart.

What I Build

Engineering domains

Backend Systems

APIs, services, authentication, databases, queues, real-time systems.

AI & LLM Applications

Document intelligence, retrieval systems, vector search, LLM integrations.

Data Engineering

Large-scale processing, batch pipelines, duplicate detection, structured data workflows.

Automation

Email automation, scraping, workflow automation, background processing.

Infrastructure

Docker, NGINX, VPS deployment, CI/CD, networking, production operations.

Selected Work

Production systems and case studies

Technology

Tools I build production systems with

Languages

  • Python
  • JavaScript
  • TypeScript
  • SQL

Backend

  • Node.js
  • Express
  • FastAPI
  • Flask
  • Async SQLAlchemy
  • REST APIs

Data

  • PostgreSQL
  • MySQL
  • MongoDB
  • Redis
  • Weaviate
  • FAISS
  • Vector Databases

AI

  • LLMs
  • Anthropic
  • Gemini
  • Ollama
  • LangChain
  • Embeddings
  • RAG
  • Document Intelligence

ML

  • Pandas
  • NumPy
  • Scikit-learn
  • Linear Regression
  • Logistic Regression
  • Random Forest

Infrastructure

  • Docker
  • Docker Compose
  • NGINX
  • AWS
  • Cloudflare
  • Portainer
  • GitHub Actions

Frontend

  • React
  • Vite
  • Redux Toolkit
  • TanStack Query
  • Material UI
  • Framer Motion

Automation & Tooling

  • Web Scraping (BeautifulSoup)
  • Multithreading
  • Microsoft Graph API
  • JSON

Engineering Philosophy

How I think about building systems

  • Build for reliability before building for scale — most systems fail from neglected edge cases, not traffic.
  • Prefer the simple architecture that works over the impressive one that might.
  • Understand a system end-to-end before changing it. Debugging what you don't understand is guessing.
  • Automate what repeats. If it's the third time by hand, it should be a script.
  • Design for failure — retries, idempotency, and recovery paths aren't optional extras.
  • Measure before optimizing. Intuition about bottlenecks is wrong more often than it's right.
  • Keep infrastructure understandable. Cleverness that only you can operate is a liability.
  • Use AI where it creates real value — grounded, retrieval-backed, and verifiable, not decorative.
  • A system that can't be deployed and maintained by someone else isn't finished.

Have a system to build, automate, or scale?