Ethel is focused on moving or updating data in content management solutions, like file systems, SharePoint, Content Server, Documentum, etc.
Ethel uses task runners to provide reusable components to perform the various parts of the updates. Task runners are organised in a job. The job details the order of the task runners and their configuration. Each task runner handles a discrete part of the job definition, such as traversing a file system, uploading a file to a target system, or adding metadata to the task definition that is needed to upload a file successfully.
Task runners are separated into three types:
extractors connect to a data source, extract the data and create tasks to be processed by transfomers and loaders. They may do actions like copying the source document to a staging location, but they should not change either the source or target systems.
transformers update the data in the task to allow the loader to complete it's work. It's expected that transformers will often be job specific components. These exist to allow extractors and loaders to be relatively static, reuseable components.
loaders take the information in the task and create or update an object in a target system.
A relatively simple job definition is:
{
"id": "Bveo3Lge8-Fg_-UZb9LV4",
"name": "Employee files production",
"runners": {
"extract": [
{
"type": "FileSystemExtractor",
"config": {
"extractRoot": "D:\\staging",
"hashName": "prod-finance-data"
}
}
],
"load": [
{ "type": "AcmeFinanceTransformer" },
{
"type": "EDMSLoader",
"config": {
"connection": {
"name": "EDMSConnections",
"key": "ProdConnection"
}
}
}
]
}
}In this example, files in the extract root directory are scanned and a task created for each file using the FileSystemExtractor task runner. The AcmeFinanceTransformer task runner then processes each file, adding metadata required by the target system to the data captured previously. Then the EDMSLoader task runner uses the data provided by the previous runners and the connection details in the configuration to connect to the target solution and upload the file.
An instance of a job is called a run. A job can have one or many runs depending on the configuration. The run contains the outcomes for each task. The run is where you find information on failed or suspended tasks as well as the time taken to complete an individual task and the various stages of the run.
Click on the Jobs and runs page for more information on jobs and runs.
Click on the Client page for more information on using the ethel client to create jobs and track runs.