JSON to Protobuf Conversion

This document describes how Xray converts human-readable JSON configuration into the internal protobuf representation used by the core engine. The conversion is a two-phase process: JSON deserialization into intermediate Go structs, followed by Build() calls that produce protobuf messages.

Overall Flow

flowchart TD
    A[Config file] --> B{Format?}
    B -->|JSON| C[DecodeJSONConfig]
    B -->|YAML| D[DecodeYAMLConfig]
    B -->|TOML| E[DecodeTOMLConfig]
    B -->|Protobuf| F[loadProtobufConfig]

    C --> G["conf.Config{}"]
    D -->|yaml->json| C
    E -->|toml->map->json| C
    F --> H["core.Config{}"]

    G --> I["conf.Config.Build()"]
    I --> H

    H --> J["core.New(config)"]
    J --> K["CreateObject() for each App"]
    K --> L[Running Xray Instance]

Phase 1: JSON to Go Structs

File: infra/conf/serial/loader.go

func DecodeJSONConfig(reader io.Reader) (*conf.Config, error) {
    jsonConfig := &conf.Config{}

    jsonReader := io.TeeReader(&json_reader.Reader{Reader: reader}, jsonContent)
    decoder := json.NewDecoder(jsonReader)

    if err := decoder.Decode(jsonConfig); err != nil {
        // Enhanced error reporting with line/char position
        var pos *offset
        switch tErr := cause.(type) {
        case *json.SyntaxError:
            pos = findOffset(jsonContent.Bytes(), int(tErr.Offset))
        case *json.UnmarshalTypeError:
            pos = findOffset(jsonContent.Bytes(), int(tErr.Offset))
        }
    }
    return jsonConfig, nil
}

The JSON reader strips comments first, then the standard encoding/json decoder fills the conf.Config struct. Sub-objects like Settings on inbound/outbound configs are kept as *json.RawMessage for deferred protocol-specific parsing.

Phase 2: Go Structs to Protobuf

The conf.Config.Build() method orchestrates the conversion. Each section follows the same pattern: validate, build the protobuf, wrap in TypedMessage.

TypedMessage Wrapping

Package: common/serial

Every protobuf message is wrapped in TypedMessage before being added to core.Config:

type TypedMessage struct {
    Type  string  // full protobuf type URL
    Value []byte  // serialized protobuf bytes
}

func ToTypedMessage(message proto.Message) *TypedMessage {
    // Marshals the message and stores its type URL
}

This is the key abstraction that allows heterogeneous config objects in a single list. The core.Config.App field is []*TypedMessage, where each entry can be any protobuf type.

CreateObject: TypedMessage to Live Object

Package: common

func CreateObject(ctx context.Context, config interface{}) (interface{}, error) {
    // Looks up the registered factory for the config's type
    // Calls the factory function to create the runtime object
}

During instance startup, core.New() iterates config.App and calls:

for _, appSettings := range config.App {
    obj, _ := CreateObject(ctx, appSettings)
    // Register the object as a feature
}

The factory was registered in each package's init():

// Example from app/dns/dns.go
func init() {
    common.Must(common.RegisterConfig((*Config)(nil), func(ctx context.Context, config interface{}) (interface{}, error) {
        return New(ctx, config.(*Config))
    }))
}

Section-by-Section Mapping

DNS Configuration

JSON:

{
    "dns": {
        "servers": [
            { "address": "8.8.8.8", "port": 53, "domains": ["geosite:google"] },
            "1.1.1.1"
        ],
        "hosts": { "example.com": "1.2.3.4" },
        "queryStrategy": "UseIP",
        "disableCache": false
    }
}

Go struct: conf.DNSConfig -> Protobuf: dns.Config

File: infra/conf/dns.go

The NameServerConfig has a custom UnmarshalJSON that accepts both a simple string ("1.1.1.1") and a full object. The Build() method:

1. Parses domain rules via parseDomainRule() which handles prefixes like geosite:, domain:, full:, regexp:, keyword:
2. Builds expectedIPs and unexpectedIPs via ToCidrList()
3. Constructs dns.NameServer protobuf with endpoint, domain rules, and GeoIP matchers
4. Computes policyID for parallel query grouping

Inbound Configuration

JSON:

{
    "inbounds": [{
        "protocol": "vless",
        "port": 443,
        "listen": "0.0.0.0",
        "settings": { "clients": [...] },
        "streamSettings": { "network": "tcp", "security": "tls" },
        "sniffing": { "enabled": true, "destOverride": ["http", "tls"] }
    }]
}

Go struct: conf.InboundDetourConfig -> Protobuf: core.InboundHandlerConfig

The build process:

func (c *InboundDetourConfig) Build() (*core.InboundHandlerConfig, error) {
    // 1. Build ReceiverConfig (listen, port, stream, sniffing)
    receiverSettings := &proxyman.ReceiverConfig{}

    // 2. Load protocol-specific config
    rawConfig, _ := inboundConfigLoader.LoadWithID(settings, c.Protocol)
    // rawConfig is e.g. *VLessInboundConfig

    // 3. Build protocol protobuf
    ts, _ := rawConfig.(Buildable).Build()
    // ts is e.g. *vless.ServerConfig (protobuf)

    // 4. Wrap both in TypedMessage
    return &core.InboundHandlerConfig{
        Tag:              c.Tag,
        ReceiverSettings: serial.ToTypedMessage(receiverSettings),
        ProxySettings:    serial.ToTypedMessage(ts),
    }, nil
}

Outbound Configuration

JSON:

{
    "outbounds": [{
        "protocol": "vless",
        "tag": "proxy",
        "settings": { "vnext": [...] },
        "streamSettings": { "network": "tcp" },
        "mux": { "enabled": true, "concurrency": 8 }
    }]
}

Go struct: conf.OutboundDetourConfig -> Protobuf: core.OutboundHandlerConfig

Build process includes:

1. SenderConfig with via address, stream settings, mux settings, and proxy chain
2. targetStrategy maps to internet.DomainStrategy enum
3. Protocol-specific settings loaded via outboundConfigLoader

API Configuration

JSON:

{
    "api": {
        "tag": "api",
        "listen": "127.0.0.1:10085",
        "services": ["HandlerService", "StatsService"]
    }
}

Go struct: conf.APIConfig -> Protobuf: commander.Config

File: infra/conf/api.go

Services are mapped by name to their protobuf config types:

func (c *APIConfig) Build() (*commander.Config, error) {
    services := make([]*serial.TypedMessage, 0)
    for _, s := range c.Services {
        switch strings.ToLower(s) {
        case "handlerservice":
            services = append(services, serial.ToTypedMessage(&handlerservice.Config{}))
        case "statsservice":
            services = append(services, serial.ToTypedMessage(&statsservice.Config{}))
        // ...
        }
    }
    return &commander.Config{Tag: c.Tag, Listen: c.Listen, Service: services}, nil
}

Routing Configuration

Go struct: conf.RouterConfig -> Protobuf: router.Config

Routing rules are parsed from JSON with support for domain lists, IP lists, port ranges, user emails, protocol names, and attributes. Each rule's Build() creates a router.RoutingRule protobuf.

The core.Config Protobuf

File: core/config.go

message Config {
    repeated InboundHandlerConfig inbound = 1;
    repeated OutboundHandlerConfig outbound = 2;
    repeated TypedMessage app = 4;
}

The App list is the core extension point. Any feature (DNS, routing, stats, observatory, reverse proxy, commander) is added here as a TypedMessage. The core iterates this list during startup and creates each feature.

Format Registration

File: core/config.go

type ConfigFormat struct {
    Name      string
    Extension []string
    Loader    ConfigLoader
}

func RegisterConfigLoader(format *ConfigFormat) error {
    configLoaderByName[name] = format
    for _, ext := range format.Extension {
        configLoaderByExt[ext] = format
    }
}

File: infra/conf/serial/builder.go

func init() {
    ReaderDecoderByFormat["json"] = DecodeJSONConfig
    ReaderDecoderByFormat["yaml"] = DecodeYAMLConfig
    ReaderDecoderByFormat["toml"] = DecodeTOMLConfig

    core.ConfigBuilderForFiles = BuildConfig
    core.ConfigMergedFormFiles = MergeConfigFromFiles
}

The BuildConfig function ties everything together:

func BuildConfig(files []*core.ConfigSource) (*core.Config, error) {
    config, _ := mergeConfigs(files)
    return config.Build()
}

Direct Protobuf Loading

Binary protobuf configs (.pb files) skip the JSON layer entirely:

func loadProtobufConfig(data []byte) (*Config, error) {
    config := new(Config)
    proto.Unmarshal(data, config)
    return config, nil
}

Only one protobuf file is allowed (no merging), and it must contain the complete core.Config.

Key Conversion Patterns

Address Types

The conf.Address type handles IP addresses, domains, and special values. Its Build() produces net.IPOrDomain:

// "1.2.3.4" -> net.IPOrDomain{Address: &IPOrDomain_Ip{Ip: [4]byte}}
// "example.com" -> net.IPOrDomain{Address: &IPOrDomain_Domain{Domain: "example.com"}}

Port Lists

conf.PortList parses ranges like "1024-65535" or comma-separated "80,443,8080" and builds net.PortList with net.PortRange entries.

String Lists

conf.StringList is a []string with custom JSON unmarshaling that accepts both a single string and an array of strings.

Domain Rules

Domain strings are parsed with prefix detection:

• "domain:example.com" -> Subdomain matching
• "full:example.com" -> Full domain matching
• "regexp:.*\\.example\\.com" -> Regex matching
• "keyword:example" -> Keyword matching
• "geosite:category" -> External geosite list
• "ext:filename:list" -> External file domain list
• No prefix -> Treated as subdomain or geosite (context-dependent)

Implementation Notes

• The JSON-to-protobuf conversion is strictly one-way. There is no facility to convert a running core.Config back to JSON (though MergeConfigFromFiles can produce merged JSON for display purposes using reflection).

• The json.RawMessage pattern for protocol settings is crucial: it allows the outer config to be parsed without knowing the protocol type, then the inner settings are parsed once the protocol is known.

• Error messages from Build() include context about which section failed (e.g., "failed to build DNS configuration"), making configuration debugging easier.

• The YAML and TOML paths both ultimately produce JSON and feed it to DecodeJSONConfig. This means all YAML/TOML configs must be representable as valid JSON structures. Complex YAML features (anchors, multi-document) may not work.

• The common.RegisterConfig / common.CreateObject system acts as a global dependency injection container. Protobuf type URLs serve as the keys, and factory functions produce the runtime objects.

• core.RequireFeatures() allows deferred initialization: a feature's factory can request other features that haven't been created yet. The core resolves these dependencies after all features are registered, triggering the deferred callbacks.