MongoDB

MongoDB is a NoSQL database that stores JSON-like documents. These documents are stored in collections, which are like database tables. The fields inside these document can differ from document to document, but generally they are all more or less the same. However documents with different schemas can be stored in the same collection.

ObjectIds

MongoDB uses the _id field as the id for a document. This has to be unique for every document in the collection. If no _id is provided when the document is created, a MongoDB ObjectId is created for that document. This id includes a timestamp, a random element and an incrementing counter, to ensure it is unique even if multiple ids are created at the same time.

The _id is often represented as:

{
  _id: ObjectId("507f1f77bcf86cd799439011")
}

To be able to transmit these ids over JSON network connections, and to use them in Lowdefy apps, Lowdefy serializes these ids as (in YAML):

_id:
  _oid: 507f1f77bcf86cd799439011

Ids specified in this way will be treated as ObjectIds by MongoDB requests and mutations.

Connections

Connection types:

• MongoDBCollection

MongoDBCollection

The MongoDBCollection connection sets up a connection to a MongoDB deployment. A connection URI with authentication credentials (username and password) is required. The URI can be in the standard or dns seedlist (srv) formats. Connections are defined on a collection level, since this allows for read/write access control on a per collection level. Access control can also be managed using the roles in the database.

Since the connection URI contains authentication secrets, it should be stored using the _secret operator.

Properties

• databaseUri: string: Required - Connection uri string for the MongoDb deployment. Should be stored using the _secret operator.
• databaseName: string: Default: Database specified in connection string - The name of the database in the MongoDB deployment.
• collection: string: Required - The name of the MongoDB collection.
• read: boolean: Default: true - Allow read operations like find on the collection.
• write: boolean: Default: false - Allow write operations like update on the collection.
• options: object: See the driver docs for more information.

Examples

MongoDB collection with reads and writes:

connections:
  - id: my_collection
    type: MongoDBCollection
    properties:
      databaseUri:
        _secret: MONGODB_URI
      collection: my_collection_name
      write: true

Environment variables:

LOWDEFY_SECRET_MONGODB_URI = mongodb+srv://username:password@server.example.com/database

Requests

Request types:

• MongoDBAggregation
• MongoDBDeleteMany
• MongoDBDeleteOne
• MongoDBFind
• MongoDBFindOne
• MongoDBInsertMany
• MongoDBInsertOne
• MongoDBUpdateMany
• MongoDBUpdateOne

MongoDBAggregation

The MongoDBAggregation request executes an aggregation pipeline in the collection specified in the connectionId. It returns the array of documents returned by the aggregation. Aggregation pipelines are MongoDB's data processing and aggregation framework. They are based on a series of stages, each of which apply a transformation to the data passed through them, like sorting, grouping or calculating additional fields.

Cursors are not supported. The request will return the whole body of the response as an array.

Properties

• pipeline: object[]: Required - Array containing all the aggregation framework commands for the execution.
• options: object: Optional settings. See the driver docs for more information. Supported settings are:
  • readPreference: string | object: The read preference.
  • allowDiskUse: boolean: Default: false - Allow disk use on the MongoDB server to store temporary results for the aggregation.
  • maxTimeMS: number: Specifies a cumulative time limit in milliseconds for processing operations on the cursor.
  • bypassDocumentValidation: boolean: Default: false - Allow driver to bypass schema validation in MongoDB 3.2 or higher.
  • collation: object: Specify collation (MongoDB 3.4 or higher) settings for update operation.
  • comment: string: Add a comment to the aggregation. These comments are visible in the MongoDB profile log, making them easier to interpret.
  • hint: string | object: Add an index selection hint to an aggregation command.

Examples

Calculate average score by region:

requests:
  - id: avg_spend_by_region
    type:  MongoDBAggregation
    connectionId: my_mongodb_collection_id
    properties:
      pipeline:
        - $group:
            _id: $region
            score:
              $avg: $score
        - $project:
            _id: 0
            region: $_id
            score: 1
        - $sort:
            score: 1

MongoDBDeleteMany

The MongoDBDeleteMany request deletes multiple documents in the collection specified in the connectionId. It requires a filter, which is written in the query syntax, to select a documents to delete.

Properties

• filter: object: Required - The filter used to select the document to update.
• options: object: Optional settings. See the driver docs for more information. Supported settings are:
  • checkKeys: boolean: Default: false - If true, will throw if bson documents start with $ or include a . in any key value.
  • collation: object: Specify collation (MongoDB 3.4 or higher) settings for update operation.
  • hint: object: An optional hint for query optimization.
  • ignoreUndefined: boolean: Default: false - Specify if the BSON serializer should ignore undefined fields.
  • j: boolean: Specify a journal write concern.
  • w: number | string: The write concern
  • wtimeout: number: The write concern timeout.

Examples

Delete all documents older than a specific date:

requests:
  - id: delete_old_documents
    type:  MongoDBDeleteMany
    connectionId: my_mongodb_collection_id
    properties:
      filter:
        created_date:
          $lt:
            _date: 2020-01-01

MongoDBDeleteOne

The MongoDBDeleteOne request deletes a single document in the collection specified in the connectionId. It requires a filter, which is written in the query syntax, to select a document to delete. It will delete the first document that matches the filter.

Properties

• filter: object: Required - The filter used to select the document to update.
• options: object: Optional settings. See the driver docs for more information. Supported settings are:
  • checkKeys: boolean: Default: false - If true, will throw if bson documents start with $ or include a . in any key value.
  • collation: object: Specify collation (MongoDB 3.4 or higher) settings for update operation.
  • hint: object: An optional hint for query optimization.
  • ignoreUndefined: boolean: Default: false - Specify if the BSON serializer should ignore undefined fields.
  • j: boolean: Specify a journal write concern.
  • w: number | string: The write concern
  • wtimeout: number: The write concern timeout.

Examples

Delete a document by _id:

requests:
  - id: delete_selected_document
    type:  MongoDBUpdateMany
    connectionId: my_mongodb_collection_id
    properties:
      filter:
        _id:
          _state: selected_id

MongoDBFind

The MongoDBFind request executes a MongoDB query on the collection specified in the connectionId. It returns the array of documents returned by the query.

Cursors are not supported. The request will return the whole body of the response as an array.

Properties

• query: object: Required - A MongoDB query object.
• options: object: Optional settings. See the driver docs for more information. Supported settings are:
  • limit: number: Sets the limit of documents returned in the query.
  • sort array | object: Set to sort the documents coming back from the query.
  • projection: object: The fields to return in the query. Object of fields to either include or exclude (one of, not both), {'a':1, 'b': 1} or {'a': 0, 'b': 0}.
  • skip: number: Set to skip N documents ahead in your query (useful for pagination).
  • hint: object: Tell the query to use specific indexes in the query. Object of indexes to use, {'_id':1}.
  • comment: string: Add a comment to the query. These comments are visible in the MongoDB profile log, making them easier to interpret.
  • readPreference: string | object: The preferred read preference.
  • maxTimeMS: number: Number of milliseconds to wait before aborting the query.
  • collation: object: Specify collation (MongoDB 3.4 or higher) settings for update operation.

Examples

Find top ten scores above 90:

requests:
  - id: scores_top_ten_scores_above_90
    type:  MongoDBFind
    connectionId: my_mongodb_collection_id
    properties:
      query:
        score:
          $gt: 90
      options:
        sort:
          - - score
            - -1
        limit: 10
        projection:
          score: 1
          name: 1

MongoDBFindOne

The MongoDBFindOne request executes a MongoDB query on the collection specified in the connectionId. It returns the first document that matches the specified query.

Cursors are not supported. The request will return the whole body of the response as an array.

Properties

• query: object: Required - A MongoDB query object.
• options: object: Optional settings. See the driver docs for more information. Supported settings are:
  • limit: number: Sets the limit of documents returned in the query.
  • sort: array | object: Set to sort the documents coming back from the query.
  • projection: object: The fields to return in the query. Object of fields to either include or exclude (one of, not both), {'a':1, 'b': 1} or {'a': 0, 'b': 0}.
  • skip: number: Set to skip N documents ahead in your query (useful for pagination).
  • hint: object: Tell the query to use specific indexes in the query. Object of indexes to use, {'_id':1}.
  • comment: string: Add a comment to the query. These comments are visible in the MongoDB profile log, making them easier to interpret.
  • readPreference: string | object: The preferred read preference.
  • maxTimeMS: number: Number of milliseconds to wait before aborting the query.
  • collation: object: Specify collation (MongoDB 3.4 or higher) settings for update operation.

Examples

Find a document by id:

requests:
  - id: find_by_id
    type:  MongoDBFindOne
    connectionId: my_mongodb_collection_id
    properties:
      query:
        _id:
          _input: _id

MongoDBInsertMany

The MongoDBInsertMany request inserts an array of documents into the collection specified in the connectionId. If a _id field is not specified on a document, a MongoDB ObjectID will be generated.

Properties

• docs: object[]: Required - The array of documents to be inserted.
• options: object: Optional settings. See the driver docs for more information. Supported settings are:
  • bypassDocumentValidation: boolean: Default: false - Allow driver to bypass schema validation in MongoDB 3.2 or higher
  • checkKeys: boolean: Default: true - If true, will throw if bson documents start with $ or include a . in any key value.
  • forceServerObjectId: boolean: Default: false - Force server to assign _id values instead of driver.
  • j: boolean: Default: false - Specify a journal write concern.
  • ignoreUndefined: boolean: Default: false - Specify if the BSON serializer should ignore undefined fields.
  • w: number | string: The write concern.
  • wtimeout: number: The write concern timeout.

Examples

Insert a set of documents:

requests:
  - id: insert_documents
    type:  MongoDBInsertMany
    connectionId: my_mongodb_collection_id
    properties:
      docs:
        - _id: 1
          value: 4
        - _id: 2
          value: 1
        - _id: 3
          value: 7

MongoDBInsertOne

The MongoDBInsertOne request inserts a document into the collection specified in the connectionId. If a _id field is not specified, a MongoDB ObjectID will be generated.

Properties

• doc: object: Required - The document to be inserted.
• options: object: Optional settings. See the driver docs for more information. Supported settings are:
  • bypassDocumentValidation: boolean: Default: false - Allow driver to bypass schema validation in MongoDB 3.2 or higher
  • checkKeys: boolean: Default: true - If true, will throw if bson documents start with $ or include a . in any key value.
  • forceServerObjectId: boolean: Default: false - Force server to assign _id values instead of driver.
  • j: boolean: Default: false - Specify a journal write concern.
  • ignoreUndefined: boolean: Default: false - Specify if the BSON serializer should ignore undefined fields.
  • w: number | string: The write concern.
  • wtimeout: number: The write concern timeout.

Examples

Insert a document:

requests:
  - id: insert_new_comment
    type:  MongoDBInsertOne
    connectionId: my_mongodb_collection_id
    properties:
      doc:
        comment:
          _state: comment_input
        user_id:
          _user: id
        timestamp:
          _date: now

MongoDBUpdateMany

The MongoDBUpdateMany request updates multiple documents that match a certain criteria in the collection specified in the connectionId. It requires a filter, which is written in the query syntax, to select the documents to update.

Properties

• filter: object: Required - The filter used to select the document to update.
• update: object | object[]: Required - The update operations to be applied to the document.
• options: object: Optional settings. See the driver docs for more information. Supported settings are:
  • arrayFilters: string[]: Array filters for the $[<identifier>] array update operator.
  • bypassDocumentValidation: boolean: Default: false - Allow driver to bypass schema validation in MongoDB 3.2 or higher.
  • checkKeys: boolean: Default: false - If true, will throw if bson documents start with $ or include a . in any key value.
  • collation: object: Specify collation (MongoDB 3.4 or higher) settings for update operation.
  • forceServerObjectId: boolean: Force server to assign _id values instead of driver.
  • hint: object: An optional hint for query optimization.
  • ignoreUndefined: boolean: Default: false - Specify if the BSON serializer should ignore undefined fields.
  • j: boolean: Specify a journal write concern.
  • upsert: boolean: Default: false - Insert document if no match is found.
  • w: number | string: The write concern.
  • wtimeout: number: The write concern timeout.

Examples

Set a list of documents as resolved:

requests:
  - id: set_resolved
    type:  MongoDBUpdateMany
    connectionId: my_mongodb_collection_id
    properties:
      # Select all documents where the _id is in selected_issues_list in state
      filter:
        _id:
          $in:
            _state: selected_issues_list
      update:
        $set:
          resolved: true

Mark all documents with score less than 6 as urgent:

requests:
  - id: set_resolved
    type:  MongoDBUpdateMany
    connectionId: my_mongodb_collection_id
    properties:
      filter:
        score:
          $lt: 6
      update:
        $set:
          status: urgent

MongoDBUpdateOne

The MongoDBUpdateOne request updates a single document in the collection specified in the connectionId. It requires a filter, which is written in the query syntax, to select a document to update. It will update the first document that matches the filter. If the upsert option is set to true, it will insert a new document if no document is found to update.

Properties

• filter: object: Required - The filter used to select the document to update.
• update: object | object[]: Required - The update operations to be applied to the document.
• options: object: Optional settings. See the driver docs for more information. Supported settings are:
  • arrayFilters: string[]: Array - Array filters for the $[<identifier>] array update operator.
  • bypassDocumentValidation: boolean: Default: false - Allow driver to bypass schema validation in MongoDB 3.2 or higher.
  • checkKeys: boolean: Default: false - If true, will throw if bson documents start with $ or include a . in any key value.
  • collation: object: Specify collation (MongoDB 3.4 or higher) settings for update operation.
  • forceServerObjectId: boolean: Force server to assign _id values instead of driver.
  • hint: object: An optional hint for query optimization.
  • ignoreUndefined: boolean: Default: false - Specify if the BSON serializer should ignore undefined fields.
  • j: boolean: Specify a journal write concern.
  • upsert: boolean: Default: false - Insert document if no match is found.
  • w: integer | string: The write concern
  • wtimeout: integer: The write concern timeout.

Examples

Update a document:

requests:
  - id: update
    type:  MongoDBUpdateOne
    connectionId: my_mongodb_collection_id
    properties:
      filter:
        _id:
          _state: _id
      update:
        $set:
          _state: true

Like a comment:

requests:
  - id: like_comment
    type:  MongoDBUpdateOne
    connectionId: my_mongodb_collection_id
    properties:
      filter:
        _id:
          _state: comments.$_id
      update:
        $inc:
          likes: 1
        $push:
          liked_by:
            _user.id:
        $set:
          last_liked:
            _date: now