workavia-drive/tdrive/backend/node

Tdrive backend

Developer guide

Getting started

1. Clone and install dependencies (assumes that you have Node.js 12 and npm installed. If not, we suggest to use nvm):

git clone git@github.com:linagora/twake-drive.git
cd twake-drive/tdrive/backend/node
npm install

2. Run in developer mode (will restart on each change)

npm run dev

3. Backend is now running and available on http://localhost:3000

Docker

Run all tests

docker-compose -f ./docker-compose.test.yml up

Run specific tests

docker-compose -f ./docker-compose.test.yml run node npm run test:unit

will run unit tests only (test:unit). For possible tests to run, check the package.json scripts.

Command Line Interface (CLI)

The Twake backend CLI

The Twake backend CLI provides a set of commands to manage/use/develop Tdrive from the twake-cli executable. Before using the CLI, you must compile Tdrive with npm run build. Once done, you can get help on on any command with the --help flag like bin/twake-cli --help.

To have prettied output when using verbose, you can use the npm script instead, (but don't forget the -- before arguments): npm run cli -- --help. You still need to compile the cli separately.

It uses the same configuration as the Tdrive backend application. Including environment variables and the ./config/default.json file.

The 'search index' command

This command re-indexes entities of the given repository from the database to the search service.

bin/twake-cli search index --repository users --repairEntities

• The --repairEntities flag means different actions for different repositories, see the output of --help for more details.

Bash completion

It's a bit awkward to do real completion from a configuration since this isn't really a node module we recommend to install globally. This should setup a shell if you're in the right path however. It's yarg's auto generated completion though; so eg: it mixes up commands and choices (or we're using it wrong).

eval "$(bin/twake-cli completion)"

Component Framework

The backend is developed using a software component approach in order to compose and adapt the platform based on needs and constraints. The current section describes this approach, and how to extend it by creating new components.

The platform has the following properties:

• A platform is composed of multiple components
• A component has an unique name in the platform
• A component can provide a service
• A component can consume services from other components
• A component has a lifecycle composed of several states: ready, initialized, started, stopped
• A component lifecycle changes when a lifecycle event is triggered by the platform: init, start, stop
• By creating links between components (service producers and consumers), components lifecycles are also linked together: A component going from ready to initialized will wait for all its dependencies to be in initialized state. This is automatically handled by the platform.

The platform currently have some limitations:

• Components can not have cyclic dependencies: if component X requires a component which requires component X directly or in one of its dependencies, the platform will not start
• Components can only have local dependencies.

Creating a new component

To create a new component, a new folder must be created under the src/services one and an index.ts file must export the a class. This class will be instantiated by the platform and will be linked to the required services automatically.

In order to illustrate how to create a component, let's create a fake Notification service.

1. Create the folder src/services/notification
2. Create an index.ts file which exports a NotificationService class

// File src/services/notification/index.ts
import { TdriveService } from "../../core/platform/framework";
import NotificationServiceAPI from "./api.ts";

export default class NotificationService extends TdriveService<NotificationServiceAPI> {
  version = "1";
  name = "notification";
  service: NotificationServiceAPI;

  api(): NotificationServiceAPI {
    return this.service;
  }
}

3. Our NotificationService class extends the generic TdriveService class and we defined the NotificationServiceAPI as its generic type parameter. It means that in the platform, the other components will be able to retrieve the component from its name and then consume the API defined in the NotificationServiceAPI interface and exposed by the api method. We need to create this NotificationServiceAPI interface which must extend the TdriveServiceProvider from the platform like:

// File src/services/notification/api.ts
import { TdriveServiceProvider } from "../../core/platform/framework/api";

export default interface NotificationServiceAPI extends TdriveServiceProvider {

  /**
   * Send a message to a list of recipients
   */
  send(message: string, recipients: string[]): Promise<string>;
}

4. Now that the interfaces are defined, we need to create the NotificationServiceAPI implementation (this is a dummy implementation which does nothing but illustrates the process):

// File src/services/notification/services/api.ts
import NotificationServiceAPI from "../api";

export class NotificationServiceImpl implements NotificationServiceAPI {
  version = "1";

  async send(message: string, recipients: string[]): Promise<string> {
    return Promise.resolve(`${message} sent`);
  }
}

5. NotificationServiceImpl now needs to be instanciated from the NotificationService class since this is where we choose to keep its reference and expose it. There are several places which can be used to instanciate it, in the constructor itself, or in one of the TdriveService lifecycle hooks. The TdriveService abstract class has several lifecycle hooks which can be extended by the service implementation for customization pusposes:

• public async doInit(): Promise<this>; Customize the init step of the component. This is generally the place where services are instanciated. From this step, you can retrieve services consumed by the current component which have been already initialized by the platform.
• public async doStart(): Promise<this>; Customize the start step of the component. You have access to all other services which are already started.

// File src/services/notification/index.ts
import { TdriveService } from "../../core/platform/framework";
import NotificationServiceAPI from "./api.ts";
import NotificationServiceImpl from "./services/api.ts";

export default class NotificationService extends TdriveService<NotificationServiceAPI> {
  version = "1";
  name = "notification";
  service: NotificationServiceAPI;

  api(): NotificationServiceAPI {
    return this.service;
  }

  public async doInit(): Promise<this> {
    this.service = new NotificationServiceImpl();

    return this;
  }
}

6. Now that the service is fully created, we can consume it from any other service in the platform. To do this, we rely on Typescript decorators to define the links between components. For example, let's say that the a MessageService needs to call the NotificationServiceAPI, we can create the link with the help of the @Consumes decorator and get a reference to the NotificationServiceAPI by calling the getProvider on the component context like:

import { TdriveService, Consumes } from "../../core/platform/framework";
import MessageServiceAPI from "./providapier";
import NotificationServiceAPI from "../notification/api";

@Consumes(["notification"])
export default class MessageService extends TdriveService<MessageServiceAPI> {

  public async doInit(): Promise<this> {
    const notificationService = this.context.getProvider<NotificationServiceAPI>("notification");

    // You can not call anything defined in the NotificationServiceAPI interface from here or from inner services by passing down the reference to notificationService.
  }
}

Configuration

The platform and services configuration is defined in the config/default.json file. It uses node-config under the hood and to configuration file inheritence is supported in the platform.

The list of services to start is defined in the services array like:

{
  "services": ["auth", "user", "channels", "webserver", "websocket", "database", "realtime"]
}

Then each service can have its own configuration block which is accessible from its service name i.e. websocket service configuration is defined in the websocket element like:

{
  "services": ["auth", "user", "channels", "webserver", "websocket", "orm"],
  "websocket": {
    "path": "/socket",
    "adapters": {
      "types": [],
      "redis": {
        "host": "redis",
        "port": 6379
      }
    }
  }
}

On the component class side, the configuration object is directly accessible from the configuration property like:

export default class WebSocket extends TdriveService<WebSocketAPI> {
  async doInit(): Promise<this> {
    // get the "path" value, defaults to "/socket" if not defined
    const path = this.configuration.get < string > ("path", "/socket");

    // The "get" method is generic and can accept custom types like
    const adapters = this.configuration.get < AdaptersConfiguration > "adapters";
  }
}

interface AdaptersConfiguration {
  types: Array<string>;
  redis: SocketIORedis.SocketIORedisOptions;
}

Platform

The Tdrive Platform is built using the component framework described just before and so, is composed of several technical services on which business services can rely on to provide a micro-services based platform.

The current chapter describes the technical services of the plaform, how to use them, how to build business services on top of them...

Current technical services are located in src/core/platform/services:

• auth: To manage authentication
• database: To manage database connections
• realtime: To provide realtime notification on platform resources
• webserver: To expose services as REST ones
• websocket: To communicate between client and server using websockets

Database Technical Service

Database technical service provides an abstraction layer over several databases to get a connection through the help of drivers and to use them in any other services.

Supported databases are currently MongoDB and Cassandra. Switching from one to other one is achieved from the database configuration document by switching the database.type flag:

{
  "database": {
    "type": "cassandra",
    "mongodb": {
      "uri": "mongodb://localhost:27017",
      "database": "tdrive"
    },
    "cassandra": {
      "contactPoints": ["localhost:9042"],
      "localDataCenter": "datacenter1",
      "keyspace": "tdrive"
    }
  }
}

In the example above, the type is set to cassandra, so the database.cassandra document will be used to connect to cassandra.

Cassandra

In order to use Cassandra, we will have to:

1. Create a keyspace. From the configuration above, the keyspace is tdrive
2. Create all the required tables

To achieve these steps, you have to use cqlsh from a terminal then:

1. Create the keyspace:

CREATE KEYSPACE tdrive WITH replication = {'class': 'NetworkTopologyStrategy', 'datacenter1': '2'} AND durable_writes = true;

2. Create the required tables

USE tdrive;

CREATE TABLE channels(company_id uuid, workspace_id uuid, id uuid, archivation_date date, archived boolean, channel_group text, description text, icon text, is_default boolean, name text, owner uuid, visibility text, PRIMARY KEY ((company_id, workspace_id), id));

MongoDB

There are no special steps to achieve to use MongoDB.