Skip to content
View Shikha-code36's full-sized avatar
💭
Databases❤️
💭
Databases❤️

Block or report Shikha-code36

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
Shikha-code36/README.md

Hi 👋 I'm Shikha Pandey

Software Engineer II @ American Express
Distributed Systems • Databases • Caching • AI Infrastructure

Portfolio • LinkedIn • Medium


About

I'm a software engineer focused on the infrastructure behind modern software systems — distributed systems, databases, caching, and AI infrastructure.

I like understanding systems from first principles:

  • How do systems behave under load?
  • Where do they stop degrading gracefully?
  • Why do failures propagate?
  • How do databases and caches behave internally?
  • What trade-offs exist between performance, correctness, and reliability?

I explore these questions through open-source contributions, reproducible experiments, systems projects, and technical writing.


🔧 Systems & Infrastructure Contributions

I contribute to production infrastructure projects, primarily around correctness, crash-safety, reliability, and performance.

Dragonfly

dragonflydb/dragonfly — high-performance in-memory datastore

My merged contributions include fixes across the core C++ engine involving:

  • Cluster slot-migration correctness
  • Production crash / SIGABRT root-cause analysis
  • Cross-shard replication and geo-query crashes
  • Sorted-set flag handling
  • Search schema validation
  • Scripting-path crashes

Several of these contributions are shipped in the v2.0.0 release.

Valkey Search

valkey-io/valkey-search

Contributed fixes involving:

  • Index-count correctness after coordinator RDB loads
  • Thread-pool failure handling
  • Shutdown and worker-state correctness

BetterDB

BetterDB-inc/monitor

Contributed reliability and security fixes around:

  • MCP server HTTP error handling
  • Internal error information exposure
  • Docker networking / DNS behavior

Additional OSS Work

I also have ongoing contributions across:

Valkey • FoundationDB • RocksDB • other distributed-systems projects

covering areas such as replication, persistence, indexing, pattern matching, and native memory management.

→ All Pull Requests
→ Dragonfly Commits
→ BetterDB Monitor Commits


🚀 Current Work

🐛 SlimyBug

Overload experimentation for distributed systems.

SlimyBug is a fault-injection and experimentation platform for studying how distributed systems behave under overload.

Instead of learning failure modes only from production incidents, SlimyBug creates controlled, reproducible experiments around:

  • Retry amplification
  • Cascading failures
  • Circuit breakers
  • Admission control
  • Connection-pool saturation
  • Overload onset
  • Dependency latency propagation
  • Retry jitter
  • Admission deferral
  • Connection-pool self-locking

What I'm investigating

Where exactly does a system stop degrading gracefully — and why?

I've run 12+ causal experiments measuring collapse boundaries and evaluating mitigation strategies. For example, a minimal circuit breaker reduced load reaching a saturated dependency by up to 44% in one experiment, followed by dedicated multi-run validation.

Stack: Python • FastAPI • PostgreSQL • Docker • Toxiproxy • Prometheus • Grafana • OpenTelemetry


🧠 SmartEvict

Learned eviction policies for semantic LLM caches.

SmartEvict investigates whether lightweight learned policies can outperform traditional cache eviction heuristics for semantic LLM workloads.

The project benchmarks a Dueling DQN policy against:

  • LRU
  • FIFO
  • GDSF

using real conversational workloads including LMSYS-Chat-1M and WildChat-1M.

📦 Repository
📖 Research Artifact / DOI


💼 Production Engineering

At American Express, I work on GenAI infrastructure and backend systems supporting internal financial-data workflows.

Some of the systems I've worked on include:

  • Custom MCP infrastructure for contextual financial data access
  • Graph-based financial data modeling using PostgreSQL + Apache AGE
  • Two-layer semantic caching using pgvector + Redis
  • LLM observability and tracing
  • Agent memory infrastructure
  • Sandboxed Python/Docker execution for data analysis
  • FastAPI microservices and production backend infrastructure

Previously at EY, I worked on enterprise NLP/LLM systems, metadata intelligence, vector retrieval, and safety infrastructure for LLM-facing systems.

Across these systems, my focus has consistently been on performance, reliability, correctness, and productionization.


🔬 Areas of Interest

Distributed Systems
Databases & Storage Engines
Caching Systems & Eviction
Replication & Consensus
Performance Engineering
Reliability Engineering
AI / LLM Infrastructure
Developer Infrastructure


🧪 Systems Projects

⚡ ArbiSim

Real-Time Cryptocurrency Arbitrage Detection System

A real-time arbitrage detection system built around a high-performance C++ engine processing high-frequency price updates across exchanges.

Stack: C++ • WebSockets • CUDA • React


🧠 Early-Exit CNN

Dynamic inference optimization using Deep Q-Learning

Implemented early-exit strategies for CNN inference using Deep Q-Learning to dynamically determine when computation can stop while maintaining model accuracy.

Stack: Python • TensorFlow • Deep Q-Learning


✍️ Writing

I write about the experiments, investigations, and engineering lessons behind my systems work.

Topics include:

Distributed Systems • Databases • Caching • AI Infrastructure • Performance • Reliability • Software Engineering

📚 Read on Medium


🛠️ Technologies

Languages & Systems

Python • C++ • Go • Distributed Systems • System Design

Databases & Messaging

PostgreSQL • Redis • MongoDB • Kafka • RabbitMQ • Elasticsearch

AI / LLM Infrastructure

OpenAI API • LangChain • MCP • RAG • Vector Search • pgvector

Backend & Infrastructure

FastAPI • Flask • Microservices • Docker • AWS • GCP • CI/CD • Prometheus • Grafana • OpenTelemetry


🌱 Currently Exploring

  • Cache internals & eviction algorithms
  • Database internals
  • Replication & consensus protocols
  • Distributed-systems failure modes
  • AI serving infrastructure
  • LLM caching
  • Performance engineering
  • Reproducible systems experimentation

📊 Engineering Philosophy

Build systems. Run experiments. Produce evidence. Share findings.

I believe the best way to understand complex systems is to build them, break them, measure them, and investigate why they behave the way they do.


📫 Connect

🌐 Portfolio • 💼 LinkedIn • 📝 Medium • 📧 Email


Build systems. Run experiments. Produce evidence. Share findings.

Pinned Loading

  1. Data-Structures-and-Algorithm-Patterns Data-Structures-and-Algorithm-Patterns Public

    Data Structures and Algorithms Patterns that I followed ,implemented in Python

    Python 117 20

  2. early-exit-cnn early-exit-cnn Public

    A deep learning framework that implements Early Exit strategies in Convolutional Neural Networks (CNNs) using Deep Q-Learning (DQN). This project enhances computational efficiency by dynamically de…

    Jupyter Notebook 8

  3. SmartEvict-Semantic-Cache-Eviction SmartEvict-Semantic-Cache-Eviction Public

    A learned, cost-aware eviction policy for semantic LLM caches that replaces heuristic policies like LRU/FIFO with a lightweight offline-trained model, improving regeneration-token savings while mai…

    Python 2

  4. assembly-ARM-tutorial assembly-ARM-tutorial Public

    A Beginner’s Guide to Assembly ARM language

    Assembly 49 12

  5. ArbiSim-A-Real-Time-Arbitrage-Detection-System ArbiSim-A-Real-Time-Arbitrage-Detection-System Public

    High-performance real-time cryptocurrency arbitrage detection system with C++ engine, WebSocket bridge, and live trading dashboard

    C++ 12 9

  6. slimybug slimybug Public

    Reproducible overload experimentation for distributed systems- find where a system stops degrading gracefully, and why.

    Python 1