VictoriaLogs Transport Server Deno Bun

This transport adds support for Victoria Metrics' VictoriaLogs and is a wrapper around the HTTP transport using the VictoriaLogs JSON stream API.

Transport Source

Vibe Code Prompts

The code has been manually tested against a local VictoriaLogs instance.

Installation

npmpnpmyarn

bash

npm install loglayer @loglayer/transport-victoria-logs serialize-error

bash

pnpm add loglayer @loglayer/transport-victoria-logs serialize-error

bash

yarn add loglayer @loglayer/transport-victoria-logs serialize-error

Basic Usage

typescript

import { LogLayer } from 'loglayer'
import { VictoriaLogsTransport } from "@loglayer/transport-victoria-logs"
import { serializeError } from "serialize-error";

const log = new LogLayer({
  errorSerializer: serializeError,
  transport: new VictoriaLogsTransport({
    url: "http://localhost:9428", // optional, defaults to http://localhost:9428
    // Configure stream-level fields for better performance
    streamFields: () => ({
      service: "my-app",
      environment: process.env.NODE_ENV || "development",
      instance: process.env.HOSTNAME || "unknown",
    }),
    // Custom timestamp function (optional)
    timestamp: () => new Date().toISOString(),
    // Custom HTTP parameters for VictoriaLogs ingestion
    httpParameters: {
      _time_field: "_time",
      _msg_field: "_msg",
    },
    // All other HttpTransport options are available and optional
    compression: false, // optional, defaults to false
    maxRetries: 3, // optional, defaults to 3
    retryDelay: 1000, // optional, defaults to 1000
    respectRateLimit: true, // optional, defaults to true
    enableBatchSend: true, // optional, defaults to true
    batchSize: 100, // optional, defaults to 100
    batchSendTimeout: 5000, // optional, defaults to 5000ms
    onError: (err) => {
      console.error('Failed to send logs to VictoriaLogs:', err);
    },
    onDebug: (entry) => {
      console.log('Log entry being sent to VictoriaLogs:', entry);
    },
  })
})

// Use the logger
log.info("This is a test message");
log.withMetadata({ userId: "123" }).error("User not found");

Configuration

The VictoriaLogs transport extends the HTTP transport configuration with VictoriaLogs specific defaults:

Option	Type	Default	Description
`url`	`string`	`"http://localhost:9428"`	The VictoriaLogs host URL. The `/insert/jsonline` path is automatically appended
`method`	`string`	`"POST"`	HTTP method to use for requests
`headers`	`Record<string, string> \| (() => Record<string, string>)`	`{ "Content-Type": "application/stream+json" }`	Headers to include in the request
`contentType`	`string`	`"application/stream+json"`	Content type for single log requests
`batchContentType`	`string`	`"application/stream+json"`	Content type for batch log requests
`streamFields`	`() => Record<string, string>`	`() => ({})`	Function to generate stream-level fields for VictoriaLogs. The keys of the returned object are automatically used as the values for the `_stream_fields` parameter. See stream fields documentation
`timestamp`	`() => string`	`() => new Date().toISOString()`	Function to generate the timestamp for the `_time` field
`httpParameters`	`Record<string, string>`	`{}`	Custom HTTP query parameters for VictoriaLogs ingestion. See HTTP parameters documentation
`payloadTemplate`	`(data: { logLevel: string; message: string; data?: Record<string, any> }) => string`	VictoriaLogs format	Pre-configured payload template for VictoriaLogs

HTTP Transport Optional Parameters

General Parameters

Name	Type	Default	Description
`enabled`	`boolean`	`true`	Whether the transport is enabled
`level`	`"trace" \| "debug" \| "info" \| "warn" \| "error" \| "fatal"`	`"trace"`	Minimum log level to process. Logs below this level will be filtered out
`method`	`string`	`"POST"`	HTTP method to use for requests
`headers`	`Record<string, string> \| (() => Record<string, string>)`	`{}`	Headers to include in the request. Can be an object or a function that returns headers
`contentType`	`string`	`"application/json"`	Content type for single log requests. User-specified headers take precedence
`compression`	`boolean`	`false`	Whether to use gzip compression
`maxRetries`	`number`	`3`	Number of retry attempts before giving up
`retryDelay`	`number`	`1000`	Base delay between retries in milliseconds
`respectRateLimit`	`boolean`	`true`	Whether to respect rate limiting by waiting when a 429 response is received
`maxLogSize`	`number`	`1048576`	Maximum size of a single log entry in bytes (1MB)
`maxPayloadSize`	`number`	`5242880`	Maximum size of the payload (uncompressed) in bytes (5MB)
`enableNextJsEdgeCompat`	`boolean`	`false`	Whether to enable Next.js Edge compatibility

Debug Parameters

Name	Type	Default	Description
`onError`	`(err: Error) => void`	-	Error handling callback
`onDebug`	`(entry: Record<string, any>) => void`	-	Debug callback for inspecting log entries before they are sent
`onDebugReqRes`	`(reqRes: { req: { url: string; method: string; headers: Record<string, string>; body: string \| Uint8Array }; res: { status: number; statusText: string; headers: Record<string, string>; body: string } }) => void`	-	Debug callback for inspecting HTTP requests and responses. Provides complete request/response details including headers and body content

Batch Parameters

Name	Type	Default	Description
`batchContentType`	`string`	`"application/json"`	Content type for batch log requests. User-specified headers take precedence
`enableBatchSend`	`boolean`	`true`	Whether to enable batch sending
`batchSize`	`number`	`100`	Number of log entries to batch before sending
`batchSendTimeout`	`number`	`5000`	Timeout in milliseconds for sending batches regardless of size
`batchSendDelimiter`	`string`	`"\n"`	Delimiter to use between log entries in batch mode
`batchMode`	`"delimiter" \| "field" \| "array"`	`"delimiter"`	Batch mode for sending multiple log entries. "delimiter" joins entries with a delimiter, "field" wraps an array of entries in an object with a field name, "array" sends entries as a plain JSON array of objects
`batchFieldName`	`string`	-	Field name to wrap batch entries in when batchMode is "field"

VictoriaLogs Specific Features

Pre-configured Payload Template

The transport comes with a pre-configured payload template that formats logs for VictoriaLogs according to the data model:

typescript

payloadTemplate: ({ logLevel, message, data }) => {
  const streamFieldsData = streamFields();
  const timeValue = timestamp();
  
  // Determine field names based on HTTP parameters
  const msgField = httpParameters._msg_field || "_msg";
  const timeField = httpParameters._time_field || "_time";
  
  return JSON.stringify({
    [msgField]: message || "(no message)",
    [timeField]: timeValue,
    level: logLevel,
    ...streamFieldsData,
    ...data,
  });
}

Note: The payload template automatically adapts to your HTTP parameters. For example, if you set _msg_field: "message" in your HTTP parameters, the transport will use message as the field name instead of _msg.

Stream Fields Configuration

Configure stream-level fields to optimize VictoriaLogs performance. Stream fields should contain fields that uniquely identify your application instance and remain constant during its lifetime:

typescript

new VictoriaLogsTransport({
  url: "http://localhost:9428",
  streamFields: () => ({
    service: "my-app",
    environment: process.env.NODE_ENV || "development",
    instance: process.env.HOSTNAME || "unknown",
    // Add other constant fields that identify your application instance
    // Avoid high-cardinality fields like user_id, ip, trace_id, etc.
  }),
})

Important:

Never add high-cardinality fields (like user_id, ip, trace_id) to stream fields as this can cause performance issues. Only include fields that remain constant during your application instance's lifetime.
The keys of the object returned by streamFields() are automatically used as the values for the _stream_fields HTTP parameter. For example, if streamFields() returns { service: "my-app", environment: "prod" }, the transport will automatically add _stream_fields=service,environment to the HTTP query parameters.

For more information about stream fields and their importance for performance, see the VictoriaLogs stream fields documentation.

HTTP Parameters

Configure custom HTTP query parameters for VictoriaLogs ingestion. This allows you to specify how VictoriaLogs should process your logs:

typescript

new VictoriaLogsTransport({
  url: "http://localhost:9428",
  httpParameters: {
    _time_field: "_time", // Specify the timestamp field name
    _msg_field: "_msg", // Specify the message field name
    // Add other VictoriaLogs HTTP parameters as needed
  },
})

Common HTTP parameters include:

_stream_fields: Comma-separated list of fields to use for stream identification (automatically set from streamFields() keys)
_time_field: Name of the timestamp field in your logs
_msg_field: Name of the message field in your logs
_default_msg_value: Default message value when the message field is empty

Important: The payload template automatically adapts to your HTTP parameters. For example:

If you set _msg_field: "message", the transport will use message as the field name instead of _msg
If you set _time_field: "timestamp", the transport will use timestamp as the field name instead of _time

For a complete list of available HTTP parameters, see the VictoriaLogs HTTP parameters documentation.

Automatic URL Construction

The transport automatically appends the /insert/jsonline path to your VictoriaLogs host URL:

typescript

// This URL: "http://localhost:9428"
// Becomes: "http://localhost:9428/insert/jsonline"
new VictoriaLogsTransport({
  url: "http://localhost:9428"
})

VictoriaLogs JSON Stream API

This transport uses the VictoriaLogs JSON stream API, which supports:

Unlimited number of log lines in a single request
Automatic timestamp handling when _time is set to "0"
Stream-based processing for high throughput
Support for custom fields and metadata

For more information about the VictoriaLogs JSON stream API, see the official documentation.

Customization

Since this transport extends the HTTP transport, you can override any HTTP transport option:

typescript

new VictoriaLogsTransport({
  url: "http://my-victoria-logs:9428",
  // Override the default payload template
  payloadTemplate: ({ logLevel, message, data }) => 
    JSON.stringify({
      _msg: message,
      _time: new Date().toISOString(),
      level: logLevel,
      custom_field: "custom_value",
      ...data,
    }),
  // Override other HTTP transport options
  compression: true,
  batchSize: 50,
  maxRetries: 5,
})

HTTP Transport - The underlying HTTP transport
VictoriaLogs Documentation - Official VictoriaLogs documentation
VictoriaLogs JSON Stream API - API documentation for the JSON stream endpoint
VictoriaLogs HTTP Parameters - HTTP query parameters documentation
VictoriaLogs Stream Fields - Stream fields documentation

VictoriaLogs Transport Server Deno Bun ​

Installation ​

Basic Usage ​

Configuration ​

HTTP Transport Optional Parameters ​

General Parameters ​

Debug Parameters ​

Batch Parameters ​

VictoriaLogs Specific Features ​

Pre-configured Payload Template ​

Stream Fields Configuration ​

HTTP Parameters ​

Automatic URL Construction ​

VictoriaLogs JSON Stream API ​

Customization ​

Related ​

VictoriaLogs Transport Server Deno Bun

Installation

Basic Usage

Configuration

HTTP Transport Optional Parameters

General Parameters

Debug Parameters

Batch Parameters

VictoriaLogs Specific Features

Pre-configured Payload Template

Stream Fields Configuration

HTTP Parameters

Automatic URL Construction

VictoriaLogs JSON Stream API

Customization

Related