Building services in TypeScript

Rockstat services are independent Node.js processes that talk to each other through Redis Pub/Sub using messages in the JSON-RPC 2.0 format. Everything shared — configuration, logging, metrics, the Redis client, RPC — is extracted into the @rockstat/rock-me-ts library (repository rockme-framework).

This guide is written around three working platform services, and the examples are built on them too:

• chwriter — writes all events to ClickHouse.
• kafkastream — sends events to Kafka topics.
• mmgeo — resolves the region by IP and parses the User-Agent.

Service roles#

Incoming requests are received by the Front service. It turns a request into an event with a routing key of the form in.gen.<service>.<name> and runs it through three phases (see Lifecycle and addressing for details):

1. Enrichment (enricher). Front sends the selected event fields to the enrich channel and waits for responses from the registered enrichers. The results are appended to the event's data. This is how mmgeo works.
2. Handling (handler). If a remote handler is registered for the event's key, Front calls it and builds the HTTP response to the client from the reply.
3. Broadcast to listeners (listener). The finished event is sent as a notification to the broadcast channel. Front does not wait for a response. This is how chwriter and kafkastream work.

Role	Channel	Call	How Front learns about the service
listener	broadcast	notification, no response	nothing needed, subscribing to the channel is enough
enricher	enrich	request, response within 200 ms	registration via __iamalive and __status
handler	service name	request, response within 200 ms	registration via __iamalive and __status

A listener is the simplest role: subscribe to broadcast and process the stream. That is where you should start a new service.

What rock-me-ts gives you#

• AppConfig — configuration from config/*.yml with EJS templates and environment variables.
• Logger — pino with a child logger per class (log.for(this)).
• Meter — StatsD metrics: tick, incr, gauge, time, timenote.
• RedisFactory and RedisClient — a Redis client built on redis-fast-driver for Pub/Sub and commands.
• RPCAdapterRedis and RPCAgnostic — transport and JSON-RPC: register, notify, request.
• TheIds — identifiers: snowflake for events, a short counter for RPC, xxhash.
• response — response builders for Front: data, redirect, pixel, error, statical.
• Message bus constants: BROADCAST, SERVICE_DIRECTOR, METHOD_IAMALIVE, METHOD_STATUS.

How a service is built#

All three services share the same skeleton. The entry point src/index.ts imports AppServer, which, on creation, brings up its dependencies and subscribes to the message bus:

import 'reflect-metadata';
import {
  AppConfig, Logger, Meter, RedisFactory,
  RPCAdapterRedis, RPCAgnostic, AgnosticRPCOptions, BROADCAST
} from '@rockstat/rock-me-ts';

export class AppServer {
  constructor() {
    const config = new AppConfig<ModuleConfig>();   // config/*.yml + .env
    const log = new Logger(config.log);
    const meter = new Meter(config.meter);
    this.log = log.for(this);

    const redisFactory = new RedisFactory({ log, meter, ...config.redis });
    const channels = [config.rpc.name, BROADCAST];  // what we listen to
    const rpcOptions: AgnosticRPCOptions = { channels, redisFactory, log, meter, ...config.rpc };

    this.rpc = new RPCAgnostic(rpcOptions);
    this.rpc.setup(new RPCAdapterRedis(rpcOptions));

    this.setup().catch(exc => { this.log.error(exc); process.exit(1); });
  }

  async setup() {
    this.rpc.register(BROADCAST, async (msg) => { /* event */ return {}; });
  }
}

export const appServer = new AppServer();

Further in this section:

• Service layout — files, package.json, tsconfig.
• Configuration — YAML, templates, environment variables.
• Message bus and RPC — channels, message format, registration.
• Event listener — chwriter and kafkastream.
• Event enricher — mmgeo.
• Build and run — dev mode, Docker, publishing the image.