DatabendWriter

Databend plugin is used to write data to Databend database via JDBC.

Databend is a database backend compatible with MySQL protocol, so Databend writing can use MySQLWriter for access.

Example

Assume the table to be written has the following DDL statement:

CREATE DATABASE example_db; CREATE TABLE example_db.table1 ( siteid INT DEFAULT CAST(10 AS INT), citycode INT, username VARCHAR, pv BIGINT );

The following configures a configuration file to read data from memory and write to databend table:

{
  "job": {
    "setting": {
      "speed": {
        "channel": 2
      }
    },
    "content": {
      "writer": {
        "name": "databendwriter",
        "parameter": {
          "preSql": [
            "truncate table @table"
          ],
          "postSql": [],
          "connection": {
            "jdbcUrl": "jdbc:databend://localhost:8000/addax",
            "table": [
              "table1"
            ]
          },
          "username": "u1",
          "password": "123",
          "column": [
            "*"
          ]
        }
      },
      "reader": {
        "name": "streamreader",
        "parameter": {
          "column": [
            {
              "random": "1,500",
              "type": "long"
            },
            {
              "random": "1,127",
              "type": "long"
            },
            {
              "value": "this is a text",
              "type": "string"
            },
            {
              "random": "5,200",
              "type": "long"
            }
          ],
          "sliceRecordCount": 100
        }
      }
    }
  }
}

Save the above configuration file as job/stream2databend.json

Execute the following command:

bin/addax.sh job/stream2Databend.json

Parameters

This plugin is based on RDBMS Writer, so you can refer to all configuration items of RDBMS Writer, and adds the following configuration items:

Configuration	Required	Type	Default Value	Description
writeMode	No	string	insert	Write mode, supports insert and replace modes
onConflictColumn	No	string	None	Conflict column, when writeMode is replace, must specify conflict column, otherwise write will fail

writeMode

Used to support Databend's replace into syntax. When this parameter is set to replace, the onConflictColumn parameter must also be specified to determine whether data is inserted or updated.

Example of both parameters:

{
  "writeMode": "replace",
  "onConflictColumn": ["id"]
}