Needless Patchouli Manufacture

    @whisklabs/grpc
    TypeScript icon, indicating that this package has built-in type declarations

    1.1.1 • Public • Published

    Protobuf generator and GRPC

    Description

    Protobuf version 3 is used to work with the backend: https://developers.google.com/protocol-buffers/docs/proto3

    Messages are sent in binary form and do not contain message structures.

    This library was created for simple convenient work with the protocol.

    It consists of 2 parts:

    • generator - converts proto files into TS code
    • query library - uses the result of the generator to send and receive requests

    Comparison

    Features Google Protobufjs pbf Current lib
    Official client + - - -
    Simple install - (binary) + (full js) + (full js) + (full js)
    Data as simple object - + + +
    Compile simples proto with imports + + - +
    GRPC client + - (poor wrapper) - (none) +
    Small client size (kb gzip) - (50 binary + http) + (6.8 binary) + (3.3 binary ) + (3.3 binary + 3.0 HTTP)
    Tree Shaking (lib + generated code) - - - +
    Dev tools (JSON) + - - +
    Dev tools (binary) - - - +
    Auto wrap StringValue - - - +
    Auto wrap oneof - (manual) - (manual) - (none) +
    Auto wrap masks - - - +

    Install

    npm i @whisklabs/grpc

    Generator

    Before using the library, you need to convert the proto files to the required format.

    API

    The conversion can be done programmatically.

    import { generator } from '@whisklabs/grpc/generator';
    
    const error = await generator({
      dir: 'source/folder', // path to root of proto folder
      out: 'result/folder', // output path for generated result
      version: '№123', // optional string for version
      exclude: /some|regexp/, // optional regexp for exclude files
      debug: true, // generate json debug files,
      messageRequired: false, // enable strict required mode for messages (default: false)
      // If we need package.json, set options packageName and packageVersion
      packageName: '@whisklabs/package-one', // generate package.json with name
      packageVersion: '0.1.10', // generate package.json with version
      packageUrl: 'git@github.com:whisklabs/npm.git', // set package url
      packageRegistry: 'https://npm.pkg.github.com/', // set package registry
    });
    
    console.log(error); // true | false

    CLI

    The conversion can be done via the command line.

    PROTO_DIR=source/folder PROTO_OUT=result/folder grpc-generator
    export PROTO_DIR=source/folder
    export PROTO_OUT=result/folder
    
    npx @whisklabs/grpc

    Required ENV params:

    • PROTO_DIR - path to root of proto folder
    • PROTO_OUT - output path for generated result
    • PROTO_VERSION - string of version (default is from npm package lib)
    • PROTO_EXCLUDE - optional regexp for exclude files
    • PROTO_DEBUG - true | false - generate json debug files
    • PROTO_MESSAGE_REQUIRED - true | false - enable strict required mode for messages (default: false)

    If we need package.json, set up both options:

    • PROTO_PACKAGE_NAME - generate package.json with name
    • PROTO_PACKAGE_VERSION - generate package.json with version
    • PROTO_PACKAGE_URL - set package url
    • PROTO_PACKAGE_REGISTRY - set package registry

    Parser

    Return JSON structure of .proto files.

    import { readFileSync } from 'fs';
    import { parser } from '@whisklabs/grpc';
    
    const parsed = parser(readFileSync('some.proto', 'utf8'));

    Issues with protoc

    For custom options add in start of file or in 'google/protobuf/descriptor.proto'

    import "google/protobuf/descriptor.proto";
    
    extend google.protobuf.FieldOptions {
      bool required = 1001; // uniq number more 1000
    }
    extend google.protobuf.MessageOptions {
      bool message_required = 1001; // uniq number more 1000
    }
    extend google.protobuf.FileOptions {
      bool messages_required = 1001; // uniq number more 1000
    }

    For optional keyword use ptotoc 3.15+

    GRPC

    The query library is a factory of endpoints, for simultaneous work with different API servers.

    Setup gRPC instance

    // Importing library
    import { grpcHTTP, StatusCode } from '@whisklabs/grpc';
    
    // Server gRPC instance
    export const grpc = grpcHTTP({
      // URL to API server [Required]
      server: 'https://example.com',
      // Use or not credentials [default = true]
      credentials: true,
      // Send binary data of request/response to grpc-web devtools [default = false]
      debug: false,
      // Send JSON data of request/response to grpc-web devtools [default = false]
      devtool: false,
      // Instance of any console. Work with combine debug or devtool [default = undefined]
      logger: console,
      // Setup global timeout in number milliseconds [default = undefined]
      timeout: undefined,
    
      // Proxy xhr before request
      transformRequest: ({ xhr, data, meta }) => {
        xhr.setRequestHeader('Authorization', 'ANY_TOKEN');
      },
    
      // Or example with modify data before send
      transformRequest: async ({ data }) => {
        if (isObject(data)) {
          (data as any).token = await SomeAction();
          return data;
        }
        return { ...data, x: 1 };
      },
    
      // Proxy result after request
      transformResponse: ({ xhr, data, meta }) => {
        if (!data.success) {
          console.log(data.error);
        }
    
        // Passthrough without changes
        return data;
      },
    
      // Proxy result after request (async example)
      transformResponse: async ({ data }) => {
        // We can globaly check errors
        if (!data.success) {
          // Notify
          alert(`v1.Grpc.Event.GRPCError ${data.error.message}`);
    
          // Prevent
          if (data.error.httpStatus === 403) {
            const reason = await forbidden();
            return { success: false, error: { message: reason } };
          }
    
          // Redirect
          if (data.error.grpcCode === StatusCode.UNAUTHENTICATED) {
            logout();
          }
        }
    
        // Passthrough without changes
        return data;
      },
    });

    Sending requests

    After configuring gRPC instance and generating code from proto files, we can make requests to the server.

    Simple example

    // Importing generated code from proto files
    import { whisk_api_user_v2_UserAPI_GetMe } from './proto';
    
    // Importing gRPC instance
    import { grpc } from './grpc';
    
    // You not need use catch for more flatten code without try {}
    const user = await grpc(whisk_api_user_v2_UserAPI_GetMe);
    
    console.log(user.success && user.data.user?.email);
    console.log(!user.success && user.error.message);
    
    if (user.success) {
      // If success we have typed data
      // You don't need do any extra check
      console.log(user.data.user?.email);
    } else {
      // If error we have same structure with optional fields
      console.log(
        user.error.message, // string?
        user.error.data, // unknown? | Object - lib tries do safe JSON.parse
        user.error.grpcCode, // number?
        user.error.httpStatus // number?
      );
    }
    
    // Destruction example for flatten style
    const { data: me, error } = await grpc(whisk_api_user_v2_UserAPI_GetMe);
    
    console.log(me?.user?.email);
    console.log(error?.message);
    
    if (me) {
      console.log(me.user?.email);
    }
    
    if (error) {
      console.log(error.message);
    }

    Cancel requests

    It can be of two types:

    • manual
    • automatic

    Example of manually canceling a request.

    // Importing
    import { grpcCancel } from '@whisklabs/grpc';
    
    // Cancel controller
    const cancel = grpcCancel();
    
    document.body.addEventListener('click', () => {
      cancel();
      console.log('gRPC canceled');
    });
    
    // Method call
    const result = await grpc(whisk_api_user_v2_UserAPI_GetMe, { cancel });
    
    if (result.success) {
      console.log(result.data.user?.id);
    }

    Example of automatic request cancellation.

    Tokens (strings) are used to identify requests. They operate within a single gRPC instance (independently of each other) and depend only on their name and nothing more.

    This is more convenient than the manual version, because occurs in fully automatic mode.

    // Start 1 request
    grpc(whisk_api_user_v2_UserAPI_GetMe, { cancel: `some-token` });
    // Stopped 1 request, because cancel ID already in progress,
    // and after this start 2 request
    grpc(whisk_api_user_v2_UserAPI_GetMe, { cancel: `some-token` });
    // Start 3 request and NOT stopped 2 request,
    // because it's another cancel ID
    grpc(whisk_api_user_v2_UserAPI_GetMe, { cancel: `some-token-${123}` });

    Mask

    Because in grpc there are no null values for collection of fields it is necessary to list all reset variables in the form of an array of lines with ways for these fields.

    import { whisk_api_user_v2_UserAPI_UpdateSettings } from './proto';
    import { mask, maskWrap } from '@whisklabs/grpc';
    
    // Manual example - the most reliable, because we do not have a single standard for masks
    const settings = { personalDetails: { age: 1 } };
    const mask = mask(settings);
    
    await grpc(whisk_api_user_v2_UserAPI_UpdateSettings, { settings, mask });
    
    // Wrapper (some projects)
    await grpc(
      whisk_api_user_v2_UserAPI_UpdateSettings,
      maskWrap({ settings: { personalDetails: { age: 1 } } }, 'settings')
      // maskWrap({ settings: { personalDetails: { age: 1 } } }, 'settings', 'otherName')
    );
    
    // Built-in wrapper (some projects)
    await grpc(
      whisk_api_user_v2_UserAPI_UpdateSettings,
      { settings: { personalDetails: { age: 1 } } },
      { mask: { field: 'settings'}
      // or with custom mask name
      // { mask: { field: 'settings', outField: 'otherName' }
    );
    
    // Semi-automatic wrapper for current object (some projects)
    await grpc(
      whisk_api_user_v2_UserAPI_UpdateSettings,
      { settings: { personalDetails: { age: 1 } } },
      { mask: true }
    );
    
    // Automatic wrapper (waiting for the standard). Future - not implemented yet!
    await grpc(
      whisk_api_user_v2_UserAPI_UpdateSettings,
      { settings: { personalDetails: { age: 1 } } }
    );

    All options

    interface Meta {
      token: string;
    }
    
    const grpc = grpcHTTP<Meta>(...);
    
    const result = await grpc(
      whisk_api_user_v2_UserAPI_UpdateSettings, // gRPC method
      { ... }, // gRPC params
      { // Query options
        mask: { field: 'settings'}, // { field: string, outField?: string } | boolean - mask
        cancel: `some-token-${123}`, // string | grpcCancel - cancel request
        onDownload: e => console.log(e.loaded / e.total), // download progress with ProgressEvent
        onUpload: e => console.log(e.loaded / e.total), // upload progress with ProgressEvent
        timeout: 2000, // number - timeout for this request with cancel request at the end
        meta: { // Meta - data for transformRequest and transformResponse methods
          token: 'CODE',
        },
      }
    );

    Web Tools

    This library is compatible with gRPC-Web Developer Tools.

    Chrome & Firefox Browser extension to aid gRPC-Web development

    Chrome extension

    Firefox extension

    grpcHTTP({
      // Send binary data of request/response to grpc-web devtools [default = false]
      debug: true,
      // Send JSON data of request/response to grpc-web devtools [default = false]
      devtool: true,
      ...
    });

    Extraction types

    import { ServiceRequest, ServiceResponse } from '@whisklabs/grpc';
    import { whisk_api_user_v2_UserAPI_UpdateSettings } from './proto';
    
    // Manual types
    type request = ServiceRequest<whisk_api_user_v2_UserAPI_UpdateSettings>;
    type response = ServiceResponse<whisk_api_user_v2_UserAPI_UpdateSettings>;

    Deep readonly

    You can switch request and response to DeepReadonly.

    import { GRPCDeep, grpcHTTP } from '@whisklabs/grpc';
    
    const grpc = grpcHTTP({ ... }) as GRPCDeep;

    Also there are helper types same as base types.

    import { FieldGetDeep, ServiceRequestDeep, ServiceResponseDeep } from '@whisklabs/grpc';

    Default values

    import { Default } from '@whisklabs/grpc';
    import { whisk_api_user_v2_TestItem } from './proto';
    
    const res = Default(whisk_api_user_v2_TestItem);
    
    // res return new default object with required fields
    // {
    //   id: '',
    //   array: [],
    //   date: { day: 0, month: 0, year: 0 },
    //   mapSearch: {},
    //   name: '',
    //   searches: [],
    // };

    Unwrap and mapping data

    import { unwrap } from '@whisklabs/grpc';
    
    const userEmail = await unwrap(
      grpc(whisk_api_user_v2_UserAPI_UpdateSettings, { id: 'abc' }),
      data => data.user?.email,
      e => {
        console.log(e);
        return 1000;
      }
    );
    
    const user = await grpc(whisk_api_user_v2_UserAPI_GetMe);
    const item = await unwrap(
      user,
      async success => {
        /* ... */
      },
      async error => {
        /* ... */
      }
    );
    
    // throw as usual promise
    const err = e => {
      throw e;
    };
    
    try {
      const request = grpc(whisk_api_user_v2_UserAPI_GetMe);
      const email = await unwrap(request, data => data.user?.email, err);
    } catch (e) {
      console.error(e);
    }

    Messages and typings

    This lib convert proto to TS, turning a dependency tree into a flat list for tree shaking.

    To ensure non-intersection, absolute paths become part of the interface name separated by "_".

    Primitives

    All primitives are required fields

    Message TypeScript Convert fn
    double number parseFloat
    float number parseFloat
    int32 number parseInt
    int64 number parseInt
    uint32 number parseInt
    uint64 number parseInt
    sint32 number parseInt
    sint64 number parseInt
    fixed32 number parseInt
    fixed64 number parseInt
    sfixed32 number parseInt
    sfixed64 number parseInt
    bool boolean === true
    string string String
    bytes Uint8Array -

    Wrappers

    Wrappers are special messages for indicate optional primitive.

    This messages auto wrap/unwrap by this lib.

    Message Alias Real code TypeScript
    google.protobuf.DoubleValue double { value: number } number?
    google.protobuf.FloatValue float { value: number } number?
    google.protobuf.Int64Value int64 { value: number } number?
    google.protobuf.UInt64Value uint64 { value: number } number?
    google.protobuf.Int32Value int32 { value: number } number?
    google.protobuf.UInt32Value uint32 { value: number } number?
    google.protobuf.BoolValue bool { value: boolean } boolean?
    google.protobuf.StringValue string { value: string } string?
    google.protobuf.BytesValue bytes { value: Uint8Array } Uint8Array?

    Messages

    Messages are same as interface in TS

    message TestItem {
      string id = 1;
      google.protobuf.StringValue description = 2;
      InnerItem item = 10
      repeated bool array = 11;
      map<string, OtherItem> map_search = 12;
    
      message InnerItem {
        int32 id = 1;
      }
    }
    
    message OtherItem {
      float count = 1;
    }
    type TestItem = {
      id: string;
      description?: string;
      item?: TestItem_InnerItem;
      array: boolean[];
      mapSearch: Record<string, OtherItem>;
    };
    
    type TestItem_InnerItem = {
      id: number;
    };
    
    type OtherItem = {
      count: number;
    };

    Enums

    As TS enum. 0 fields name with ending _INVALID or _UNSPECIFIED removed from typings.

    message TestItem {
      Type id = 1;
      Type force = 2 [ required = true ];
      Direction dir = 3;
    }
    
    enum Type {
      TYPE_UNSPECIFIED = 0; // removed
      HEIGHT = 1;
      WIDTH = 2;
    }
    
    enum Direction {
      UP = 0; // Not used in our proto, but can be in third party proto
      DOWN = 1;
    }
    type TestItem = {
      id?: Type;
      forced: Type;
      dir?: Direction;
    };
    
    const enum Type {
      HEIGHT = 1,
      WIDTH = 2,
    }
    
    const enum Direction {
      UP = 0,
      DOWN = 1,
    }

    Services

    This is information about server methods, what types of messages are used for request and response.

    message Request {
      int32 id = 1;
    }
    
    message Response {
      string name = 1;
    }
    
    service UserAPI {
      rpc GetMe(Request) returns (Response) {
        // can be some options
      }
    }
    type Request = {
      id: number;
    };
    
    type Response = {
      name: string;
    };
    
    export type UserAPI_GetMe = Service<Field<Request>, Field<Response>>;
    
    // get back
    
    type request = ServiceRequest<UserAPI_GetMe>;
    type response = ServiceResponse<UserAPI_GetMe>;

    Oneof

    This is a grouping of fields where only one of them can be set or read.

    // file path: whisk/api/user/v2/user.proto
    message TestOneof {
      string id = 1;
    
      oneof device {
        EthicalPreference device_type = 11;
        string custom_device = 12;
      }
    }
    export type whisk_api_user_v2_TestOneof = {
      id: string;
      device?:
        | { oneof: 'deviceType'; value: whisk_api_user_v2_EthicalPreference }
        | { oneof: 'customDevice'; value: string };
    };

    There are two helpers for work with oneof: oneof and oneis.

    const data: whisk_api_user_v2_TestOneof = {
      id: '123',
      device: {
        oneof: 'customDevice',
        value: 'abc',
      },
    };
    
    // any value
    const any = oneof(data.device);
    // or
    const val = data.device?.value;
    
    // oneis example
    if (oneis(data.device, 'customDevice')) {
      console.log(data.device?.value);
    }
    
    // If example
    if (data.device?.oneof === 'customDevice') {
      console.log(data.device?.value);
    }
    
    // Limit value variants
    const a = oneof(data.device, 'customDevice') ?? oneof(data.device, 'deviceType');
    console.log(a);
    
    // Function transform
    const out = oneof(data.device, 'customDevice', v => `${v}a`) ?? oneof(data.device, 'deviceType', v => v * 3);
    console.log(out);
    
    // Switch example
    let res: string | number | undefined;
    switch (data.device?.oneof) {
      case 'customDevice':
        res = `${data.device.value}a`;
        break;
      case 'deviceType':
        res = data.device.value * 3;
        break;
    }
    console.log(res);

    Required and Optional

    By default all primitives, repeated and map are required. Other types are optional.

    You can change this behaveour using option required or keyword optional.

    message TestOneof {
      string id = 1;
      string name = 2 [ required = true ];
      string item = 3 [ required = false ];
    
      google.protobuf.StringValue description = 10;
      google.protobuf.StringValue test = 11 [ required = true ];
      google.protobuf.StringValue result = 12 [ required = false ];
      optional string story = 13;
    
      whisk.api.shared.v1.Time time = 30;
      whisk.api.shared.v1.Date date = 31 [ required = true ];
      whisk.api.shared.v1.Date date_new = 32 [ (required) = true ]; // ptotoc compatable
    }
    export type whisk_api_user_v2_TestOneof = {
      id: string;
      name: string;
      item?: string;
    
      description?: string;
      test: string;
      result?: string;
      story?: string;
    
      time?: whisk_api_shared_v1_Time;
      date: whisk_api_shared_v1_Date;
      dateNew: whisk_api_shared_v1_Date;
    };

    You can switch on strict required mode for messages with optional keyword:

    • All fields are required by default, expect oneof.
    • To mark a field as optional, you can add the keyword optional.
    • Although this keyword has no effect on binary compatibility and can be added or removed, clients need to be updated so that they understand how to handle values correctly.
    • For backward compatibility, wrappers can be used because they are not binary compatible with optional.
    • If you cannot change the structure in any way, but you need to force the override, then in such a case it is permissible to use [(required) = true] or [(required) = false].
    1. Global

      Work on all files in final

      PROTO_MESSAGE_REQUIRED=true npx @whisklabs/grpc@1
      import { generator } from '@whisklabs/grpc/generator';
      
      const error = await generator({
        messageRequired: true, // enable strict required mode for messages (default: false)
      });
    2. Local per file

      Add option in start of proto file

      syntax = "proto3";
      
      package whisk.api.user.v2;
      
      // Force required mode for messages in file
      option (messages_required) = true; // false for disable
      
      message Test {}
    3. Local per message

      message Day {
        int32 num = 1;
      }
      
      message Week {
        // Force required mode in message
        option (message_required) = true; // false for disable
      
        int32 num = 1;
        Day day = 2;
      }

    Example:

    message Test {
      // Primitive
      string id = 1; // required
      optional string text = 2; // optional
    
      // Messages
      Week current_week = 11; // required
      optional Week next_week = 12; // optional
      whisk.api.shared.v1.Time time = 13; // required
      optional whisk.api.shared.v1.Time time_after = 14; // optional
    
      // Wrappers (legacy)
      google.protobuf.StringValue description = 21; // optional
    
      // Force override (backward binary compatibility only)
      string item = 31 [ (required) = false ]; // optional
      google.protobuf.StringValue test = 32 [ (required) = true ]; // required
    
      // Repeated - can't work with optional!
      repeated bool array = 41; // required
      repeated bool array_2 = 42 [ (required) = false ]; // optional
    
      // Map - can't work with optional!
      map<string, bool> map_search = 51; // required
      map<string, bool> map_search_2 = 52 [ (required) = false ]; // optional
    
      // Oneof - can't work with optional!
      oneof device_description {
        DeviceType device_type = 61; // required
        DeviceType custom_device = 62 [ (required) = false ]; // optional
      }
    }
    
    message Week {
      int32 num = 1;
    }

    Result:

    type Test = {
      id: string;
      text?: string;
      currentWeek: Week;
      nextWeek?: Week;
      time: whisk_api_shared_v1_Time;
      timeAfter?: whisk_api_shared_v1_Time;
      description: string;
      item?: string;
      test: string;
      array: boolean[];
      array_2?: boolean[];
      mapSearch: Record<string, boolean>;
      mapSearch_2?: Record<string, boolean>;
      deviceDescription?:
        | { oneof: 'deviceType'; value: Device_DeviceType }
        | { oneof: 'customDevice'; value?: Device_DeviceType };
    };

    Deprecated

    For obsolete fields, you can mark them as deprecated.

    message TestOneof {
      string id = 1 [ deprecated = true ];
      google.protobuf.StringValue description = 10 [ required = true, deprecated = true ];
    }
    export type whisk_api_user_v2_TestOneof = {
      /**  @deprecated true */
      id: string;
      /**  @deprecated true */
      description: string;
    };

    Defaults

    If field is required, bit not present in message, it is installed by table:

    Message Default
    double 0
    float 0
    int32 0
    int64 0
    uint32 0
    uint64 0
    sint32 0
    sint64 0
    fixed32 0
    fixed64 0
    sfixed32 0
    sfixed64 0
    enum 0
    bool false
    string ''
    bytes Uint8Array

    Install

    npm i @whisklabs/grpc

    DownloadsWeekly Downloads

    75

    Version

    1.1.1

    License

    MIT

    Unpacked Size

    621 kB

    Total Files

    280

    Last publish

    Collaborators

    • viktor-whisk
    • a.k.samsungnext