Overview

Flexible and extensible storage system for Vercube applications

The Storage module provides a powerful, flexible storage system for Vercube applications. Built around a provider-based architecture, it allows you to store and retrieve data across multiple storage backends through a unified interface. Whether you need in-memory caching, file system storage, or cloud-based solutions, the Storage module handles it seamlessly.

Installation

$ pnpm add @vercube/storage

Quick Start

Register StorageManager in Container

Set up the StorageManager in your DI container. This is typically done once during application bootstrap.

src/container.ts
import { Container } from '@vercube/di';
import { StorageManager } from '@vercube/storage';
import { MemoryStorage } from '@vercube/storage/drivers/MemoryStorage';

export function setupContainer(container: Container): void {
  // Bind StorageManager to the container
  container.bind(StorageManager);
  
  // Get the storage manager instance
  const storageManager = container.get(StorageManager);
  
  // Mount a memory storage instance
  storageManager.mount({
    name: 'cache',
    storage: MemoryStorage
  });
}

Use Storage in Services

Use dependency injection to access storage in your services. Store and retrieve data for caching, session management, or any other purpose.

src/services/UserService.ts
import { Inject } from '@vercube/di';
import { StorageManager } from '@vercube/storage';

export class UserService {
  @Inject(StorageManager)
  private storageManager!: StorageManager;
  
  async getUserProfile(userId: string) {
    const cacheKey = `user:${userId}`;
    
    // Try to get from cache first
    const cached = await this.storageManager.getItem<UserProfile>({
      storage: 'cache',
      key: cacheKey
    });
    if (cached) {
      return cached;
    }
    
    // Fetch from database
    const user = await this.database.findUser(userId);
    
    // Cache for future requests
    await this.storageManager.setItem({
      storage: 'cache',
      key: cacheKey,
      value: user
    });
    
    return user;
  }
  
  async updateUserProfile(userId: string, data: UpdateUserDto) {
    // Update in database
    const user = await this.database.updateUser(userId, data);
    
    // Invalidate cache
    await this.storageManager.deleteItem({
      storage: 'cache',
      key: `user:${userId}`
    });
    
    return user;
  }
}

Use Storage in Controllers

Inject the StorageManager into controllers for request-level caching or rate limiting.

src/controllers/ProductController.ts
import { Controller, Get, Post } from '@vercube/core';
import { Inject } from '@vercube/di';
import { StorageManager } from '@vercube/storage';

@Controller('/products')
export class ProductController {
  @Inject(StorageManager)
  private storageManager!: StorageManager;
  
  @Inject(ProductService)
  private productService!: ProductService;
  
  @Get('/')
  async getProducts() {
    const cacheKey = 'products:all';
    
    // Check cache
    const cached = await this.storageManager.getItem<Product[]>({
      storage: 'cache',
      key: cacheKey
    });
    if (cached) {
      return Response.json(cached);
    }
    
    // Fetch and cache
    const products = await this.productService.findAll();
    await this.storageManager.setItem({
      storage: 'cache',
      key: cacheKey,
      value: products
    });
    
    return Response.json(products);
  }
  
  @Get('/:id')
  async getProduct(req: Request, params: { id: string }) {
    const cacheKey = `products:${params.id}`;
    
    const cached = await this.storageManager.getItem<Product>({
      storage: 'cache',
      key: cacheKey
    });
    if (cached) {
      return Response.json(cached);
    }
    
    const product = await this.productService.findById(params.id);
    if (product) {
      await this.storageManager.setItem({
        storage: 'cache',
        key: cacheKey,
        value: product
      });
    }
    
    return Response.json(product);
  }
}

Core Concepts

StorageManager

The StorageManager is the central service for managing storage instances. It handles mounting and accessing storage backends. Think of it as a registry for all your storage providers.

import { StorageManager } from '@vercube/storage';
import { MemoryStorage } from '@vercube/storage/drivers/MemoryStorage';

const storageManager = container.get(StorageManager);

// Mount storage backends
storageManager.mount({ name: 'cache', storage: MemoryStorage });
storageManager.mount({ name: 'sessions', storage: MemoryStorage });

// Use storage operations directly through StorageManager
await storageManager.setItem({ storage: 'cache', key: 'foo', value: 'bar' });
const value = await storageManager.getItem({ storage: 'cache', key: 'foo' });

// Or get a storage instance directly
const cache = storageManager.getStorage('cache');

Storage Interface

The Storage abstract class defines the interface for all storage implementations. Each storage backend must implement these core methods:

  • getItem() - Retrieve a value by key
  • setItem() - Store a value with a key
  • hasItem() - Check if a key exists
  • deleteItem() - Delete a value by key
  • getKeys() - List all keys
  • clear() - Remove all items
  • size() - Get the number of stored items

Type Safety

Vercube Storage is fully type-safe. You can specify the expected type when retrieving data:

interface UserProfile {
  id: string;
  name: string;
  email: string;
}

// Type-safe retrieval
const user = await storage.getItem<UserProfile>('user:123');
// user is UserProfile | null

// Works with arrays too
const users = await storage.getItem<UserProfile[]>('users:all');

IOC Container Integration

The Storage module integrates seamlessly with Vercube's dependency injection system. You register the StorageManager in the IOC container and inject it wherever needed using the @Inject decorator.

Configuration

Basic Configuration

Configure storage when setting up your application:

import { Container } from '@vercube/di';
import { StorageManager } from '@vercube/storage';
import { MemoryStorage } from '@vercube/storage/drivers/MemoryStorage';

export function setupContainer(container: Container): void {
  container.bind(StorageManager);
  
  const storageManager = container.get(StorageManager);
  
  // Mount with basic configuration
  storageManager.mount({
    name: 'cache',
    storage: MemoryStorage
  });
}

Multiple Storage Backends

You can mount multiple storage backends for different use cases:

import { StorageManager } from '@vercube/storage';
import { MemoryStorage } from '@vercube/storage/drivers/MemoryStorage';

const storageManager = container.get(StorageManager);

// Fast in-memory cache for frequently accessed data
storageManager.mount({
  name: 'cache',
  storage: MemoryStorage
});

// Separate storage for user sessions
storageManager.mount({
  name: 'sessions',
  storage: MemoryStorage
});

// Storage for temporary data
storageManager.mount({
  name: 'temp',
  storage: MemoryStorage
});

Environment-Specific Configuration

Configure different storage backends for different environments:

import { StorageManager } from '@vercube/storage';
import { MemoryStorage } from '@vercube/storage/drivers/MemoryStorage';

const isDevelopment = process.env.NODE_ENV === 'development';
const isProduction = process.env.NODE_ENV === 'production';

const storageManager = container.get(StorageManager);

if (isDevelopment) {
  // Use memory storage in development
  storageManager.mount({
    name: 'cache',
    storage: MemoryStorage
  });
} else {
  // Use persistent storage in production
  storageManager.mount({
    name: 'cache',
    storage: MemoryStorage, // Replace with your production storage
    initOptions: {
      // Production-specific options
    }
  });
}

Storage Operations

Basic CRUD Operations

You can use StorageManager methods directly with object parameters:

// Create/Update - setItem
await storageManager.setItem({
  storage: 'cache',
  key: 'user:123',
  value: {
    id: '123',
    name: 'John Doe',
    email: '[email protected]'
  }
});

// Read - getItem
const user = await storageManager.getItem<User>({
  storage: 'cache',
  key: 'user:123'
});

// Check existence - hasItem
const exists = await storageManager.hasItem({
  storage: 'cache',
  key: 'user:123'
});

// Delete - deleteItem
await storageManager.deleteItem({
  storage: 'cache',
  key: 'user:123'
});

Or get a storage instance and use it directly:

const storage = storageManager.getStorage('cache');

await storage.setItem('user:123', { name: 'John' });
const user = await storage.getItem<User>('user:123');
await storage.deleteItem('user:123');

Working with Keys

// Get all keys from storage
const allKeys = await storageManager.getKeys({ storage: 'cache' });
console.log(allKeys); // ['user:1', 'user:2', 'product:1', ...]

// Filter keys by prefix manually
const userKeys = allKeys.filter(key => key.startsWith('user:'));
console.log(userKeys); // ['user:1', 'user:2', ...]

// Get storage size
const count = await storageManager.size({ storage: 'cache' });
console.log(count); // 5

// Clear all items
await storageManager.clear({ storage: 'cache' });

Advanced Patterns

Cache-Aside Pattern

Implement cache-aside (lazy-loading) pattern for database queries:

export class ProductRepository {
  @Inject(StorageManager)
  private storageManager!: StorageManager;
  
  @Inject(Database)
  private database!: Database;
  
  async findById(id: string): Promise<Product | null> {
    const cacheKey = `product:${id}`;
    
    // 1. Check cache first
    const cached = await this.storageManager.getItem<Product>({
      storage: 'cache',
      key: cacheKey
    });
    if (cached) {
      return cached;
    }
    
    // 2. Cache miss - fetch from database
    const product = await this.database.products.findById(id);
    
    // 3. Store in cache for next time
    if (product) {
      await this.storageManager.setItem({
        storage: 'cache',
        key: cacheKey,
        value: product
      });
    }
    
    return product;
  }
  
  async update(id: string, data: UpdateProductDto): Promise<Product> {
    // Update database
    const product = await this.database.products.update(id, data);
    
    // Invalidate cache
    await this.storageManager.deleteItem({
      storage: 'cache',
      key: `product:${id}`
    });
    
    return product;
  }
}

Cache Warming

Pre-populate cache with frequently accessed data:

export class CacheWarmingService {
  @Inject(StorageManager)
  private storageManager!: StorageManager;
  
  @Inject(ProductService)
  private productService!: ProductService;
  
  async warmProductCache(): Promise<void> {
    // Fetch popular products
    const popularProducts = await this.productService.findPopular(100);
    
    // Cache each product
    for (const product of popularProducts) {
      await this.storageManager.setItem({
        storage: 'cache',
        key: `product:${product.id}`,
        value: product
      });
    }
    
    console.log(`Warmed cache with ${popularProducts.length} products`);
  }
}

Cache Invalidation Patterns

export class CacheInvalidationService {
  @Inject(StorageManager)
  private storageManager!: StorageManager;
  
  // Invalidate single item
  async invalidateProduct(id: string): Promise<void> {
    await this.storageManager.deleteItem({
      storage: 'cache',
      key: `product:${id}`
    });
  }
  
  // Invalidate by pattern (all products)
  async invalidateAllProducts(): Promise<void> {
    const allKeys = await this.storageManager.getKeys({ storage: 'cache' });
    const productKeys = allKeys.filter(key => key.startsWith('product:'));
    
    for (const key of productKeys) {
      await this.storageManager.deleteItem({ storage: 'cache', key });
    }
  }
  
  // Invalidate related items
  async invalidateCategory(categoryId: string): Promise<void> {
    // Remove category
    await this.storageManager.deleteItem({
      storage: 'cache',
      key: `category:${categoryId}`
    });
    
    // Remove all products in category
    const allKeys = await this.storageManager.getKeys({ storage: 'cache' });
    const productKeys = allKeys.filter(key => 
      key.startsWith(`category:${categoryId}:products`)
    );
    
    for (const key of productKeys) {
      await this.storageManager.deleteItem({ storage: 'cache', key });
    }
  }
}

Rate Limiting with Storage

export class RateLimiter {
  @Inject(StorageManager)
  private storageManager!: StorageManager;
  
  private readonly maxRequests = 100;
  
  async isRateLimited(clientId: string): Promise<boolean> {
    const key = `ratelimit:${clientId}`;
    
    const current = await this.storageManager.getItem<number>({
      storage: 'cache',
      key
    }) || 0;
    
    if (current >= this.maxRequests) {
      return true;
    }
    
    await this.storageManager.setItem({
      storage: 'cache',
      key,
      value: current + 1
    });
    
    return false;
  }
}

Session Management

interface Session {
  userId: string;
  createdAt: number;
  data: Record<string, unknown>;
}

export class SessionService {
  @Inject(StorageManager)
  private storageManager!: StorageManager;
  
  async createSession(userId: string): Promise<string> {
    const sessionId = crypto.randomUUID();
    
    const session: Session = {
      userId,
      createdAt: Date.now(),
      data: {}
    };
    
    await this.storageManager.setItem({
      storage: 'sessions',
      key: `session:${sessionId}`,
      value: session
    });
    
    return sessionId;
  }
  
  async getSession(sessionId: string): Promise<Session | null> {
    return this.storageManager.getItem<Session>({
      storage: 'sessions',
      key: `session:${sessionId}`
    });
  }
  
  async destroySession(sessionId: string): Promise<void> {
    await this.storageManager.deleteItem({
      storage: 'sessions',
      key: `session:${sessionId}`
    });
  }
  
  async updateSessionData(
    sessionId: string, 
    data: Record<string, unknown>
  ): Promise<void> {
    const session = await this.getSession(sessionId);
    
    if (session) {
      session.data = { ...session.data, ...data };
      await this.storageManager.setItem({
        storage: 'sessions',
        key: `session:${sessionId}`,
        value: session
      });
    }
  }
}
Report an issue or Edit this page on GitHub
Previous

Drivers

Built-in storage providers and guide to creating custom drivers

Next