Oracle Reader
Oracle Reader plugin is used to read data from Oracle.
Configuration Example
Configure a job to synchronize and extract data from Oracle database to local:
json
{
"job": {
"setting": {
"speed": {
"byte": 1048576,
"channel": 1
}
},
"content": {
"reader": {
"name": "oraclereader",
"parameter": {
"username": "oracle",
"password": "password",
"column": [
"id",
"name"
],
"splitPk": "db_id",
"connection": {
"table": [
"table"
],
"jdbcUrl": "jdbc:oracle:thin:@127.0.0.1:5432/orcl"
}
}
},
"writer": {
"name": "streamwriter",
"parameter": {
"print": true
}
}
}
}
}Parameters
This plugin is based on RDBMS Reader, so you can refer to all configuration items of RDBMS Reader.
Support for GEOMETRY Type
Starting from Addax 4.0.13, experimental support for Oracle GEOMETRY type is provided. This plugin converts this type of data to JSON array strings.
Suppose you have such a table and data:
sql
-- create demo table
CREATE TABLE spatial_data(id int, name varchar2(50), geo SDO_GEOMETRY);
-- insert some records
INSERT INTO HR.spatial_data (id, name, geo) VALUES (
1,
'Point 1',
MDSYS.SDO_GEOMETRY(
2001, -- 二维点类型
NULL,
MDSYS.SDO_POINT_TYPE(1, 1, NULL), -- X, Y 坐标为 1
NULL,
NULL
)
);
INSERT INTO HR.spatial_data (id, name, geo) VALUES (
2,
'Line 1',
MDSYS.SDO_GEOMETRY(
2002, -- 二维线类型
NULL,
NULL,
MDSYS.SDO_ELEM_INFO_ARRAY(1,2,1), -- 表示一个由两个点组成的线段
MDSYS.SDO_ORDINATE_ARRAY(1,1, 2,2) -- 线段的起点为 (1,1)，终点为 (2,2)
)
);
INSERT INTO HR.spatial_data (id, name, geo) VALUES (
3,
'Polygon 1',
MDSYS.SDO_GEOMETRY(
2003, -- 二维多边形类型
NULL,
NULL,
MDSYS.SDO_ELEM_INFO_ARRAY(1,1003,1), -- 表示一个由三个点组成的三角形
MDSYS.SDO_ORDINATE_ARRAY(1,1, 2,2, 3,1) -- 三角形的三个顶点分别为 (1,1)、(2,2) 和 (3,1)
)
);The final output result of reading this table data is similar to the following:
txt
___ _ _
/ _ \ | | | |
/ /_\ \ __| | __| | __ ___ __
| _ |/ _` |/ _` |/ _` \ \/ /
| | | | (_| | (_| | (_| |> <
\_| |_/\__,_|\__,_|\__,_/_/\_\
:: Addax version :: (v4.0.13-SNAPSHOT)
2023-04-24 22:45:03.270 [ main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-04-24 22:45:03.285 [ main] INFO Engine -
{
"content":{
"reader":{
"name":"oraclereader",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":[
"jdbc:oracle:thin:@127.0.0.1:1521/XE"
],
"table":[
"HR.spatial_data"
]
}
],
"username":"oracle",
"password":"*****"
}
},
"writer":{
"name":"streamwriter",
"parameter":{
"print":true
}
}
},
"setting":{
"speed":{
"bytes":-1,
"channel":1
}
}
}
2023-04-24 22:45:03.305 [ main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false
2023-04-24 22:45:03.305 [ main] INFO JobContainer - Addax jobContainer starts job.
2023-04-24 22:45:03.306 [ main] INFO JobContainer - Set jobId = 0
2023-04-24 22:45:04.540 [ job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:oracle:thin:@127.0.0.1:1521/XE.
2023-04-24 22:45:04.541 [ job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-04-24 22:45:04.545 [ job-0] INFO JobContainer - Addax Reader.Job [oraclereader] do prepare work .
2023-04-24 22:45:04.546 [ job-0] INFO JobContainer - Addax Writer.Job [streamwriter] do prepare work .
2023-04-24 22:45:04.546 [ job-0] INFO JobContainer - Job set Channel-Number to 1 channel(s).
2023-04-24 22:45:04.548 [ job-0] INFO JobContainer - Addax Reader.Job [oraclereader] splits to [1] tasks.
2023-04-24 22:45:04.548 [ job-0] INFO JobContainer - Addax Writer.Job [streamwriter] splits to [1] tasks.
2023-04-24 22:45:04.569 [ job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2023-04-24 22:45:04.577 [ taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2023-04-24 22:45:04.579 [ taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-04-24 22:45:04.580 [ taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2023-04-24 22:45:04.593 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from HR.spatial_data
] jdbcUrl:[jdbc:oracle:thin:@10.90.70.11:13521/XE].
2023-04-24 22:45:05.561 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from HR.spatial_data
] jdbcUrl:[jdbc:oracle:thin:@10.90.70.11:13521/XE].
1 Point 1 [{"sdo_gtype":2001,"sdo_srid":0,"sdo_point":{"x":1.0,"y":1.0,"z":null}}]
2 Line 1 [{"sdo_gtype":2002,"sdo_srid":0,"sdo_point":{"x":null,"y":null,"z":null},"sdo_elem_info":[1,2,1],"sdo_ordinates":[1.0,1.0,2.0,2.0]}]
3 Polygon 1 [{"sdo_gtype":2003,"sdo_srid":0,"sdo_point":{"x":null,"y":null,"z":null},"sdo_elem_info":[1,1003,1],"sdo_ordinates":[1.0,1.0,2.0,2.0,3.0,1.0]}]
2023-04-24 22:45:07.589 [ job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2023-04-24 22:45:07.589 [ job-0] INFO JobContainer - Addax Writer.Job [streamwriter] do post work.
2023-04-24 22:45:07.590 [ job-0] INFO JobContainer - Addax Reader.Job [oraclereader] do post work.
2023-04-24 22:45:07.592 [ job-0] INFO JobContainer - PerfTrace not enable!
2023-04-24 22:45:07.593 [ job-0] INFO StandAloneJobContainerCommunicator - Total 3 records, 372 bytes | Speed 124B/s, 1 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-04-24 22:45:07.594 [ job-0] INFO JobContainer -
任务启动时刻 : 2023-04-24 22:45:03
任务结束时刻 : 2023-04-24 22:45:07
任务总计耗时 : 4s
任务平均流量 : 124B/s
记录写入速度 : 1rec/s
读出记录总数 : 3
读写失败总数 : 0Note: This data type is currently in experimental support stage. The author's understanding of this data type is not deep, and it has not been comprehensively tested. Please do not use it directly in production environments.