GUI Application Guide

The DaRT Reader window lets you:

• open a folder of DaRT extracts and see what each one holds
• check that an extract's payload files are whole
• look at the records of a segment before writing anything
• write segments to a database or to CSV files, and check what was written

The project window

DaRT Reader opens one window, the project window. It has three panes side by side:

• Project, on the left: the files of the folder you opened.
• The extract, in the middle: what the directory file you clicked says about the extract, and whether its payload files are whole. The two write buttons are here.
• Segments, on the right: every segment of the extract, to look at and to tick for writing.

The menus are File, View, Run and Help. On macOS they stand in the menu bar at the top of the screen, on Windows in the window itself.

Making a pane wider or narrower

Point at the line between two panes until the pointer becomes a double arrow, then drag it left or right. A pane cannot be dragged narrower than its contents need. When you resize the window, the middle pane takes the difference. The widths are not kept: the next start opens at the usual widths again.

Entering the license key

You need a license key to open an extract. Choose File > Settings…, enter the key under Licence key and click Save. See Changing the settings.

Opening an extract

Make sure that the directory file and the payload files of an extract are in the same folder. Then choose File > Open project… and pick that folder. Its files appear in the Project pane.

Directory files — their names end in _DR — are shown in bold, and only they can be opened. Click one. The payload files that belong to it are marked PAYLOAD, and the extract appears in the middle pane. File > Close project empties the window again.

Opening takes a moment, mostly for the licensing service to answer. Meanwhile the middle pane shows the file's name, and if it takes longer than a glance, a spinner and what it is waiting for — for example Opening t1_DR — asking the licensing service…. If you click another directory file in the meantime, that one is opened and the first is forgotten.

Below the file list, Open any file lets you click files that are not directory files as well. The middle pane then says what the file is instead, for example This file is not a directory file.

What the middle pane shows

At the top, the name of the directory file and a verdict on its payload files: Complete and consistent when every payload file is there, was written to its end, measures the size it states and comes from the same run of DaRT as the directory file — or Something is wrong with a payload file when one of them is not. This says that the files are whole, not that everything in them can be read.

Under The extract you see what the extract states about itself: the logical system, the DaRT and SAP release, the user who requested it, when it was requested, the fiscal year, the code page and how it was written.

Under Payload files, each payload file with the size it states in the record it ends with, the size it actually has on disk, and the difference between the two, which has to read 0.

Without a license key

An extract is only shown once the licensing service has confirmed your key. The service is asked every time a directory file is opened, so this needs a connection to the internet; the key is not checked on your machine, and offline no extract can be opened.

Until the key is confirmed, the middle pane reads Not opened: there is no confirmed license, with the reason beneath it and a Settings… button for entering the key. Nothing of the extract is shown — no summary, no segments, no fields, no record counts — because nothing of it has been read. What you do see is about the files: that you clicked a DaRT directory file, and whether its payload files are whole. The tree still marks which files belong to the extract.

The reasons are listed in Error messages.

After entering the key, click the directory file in the tree again. A demonstration key is a key too: the extract opens with all its segments and fields.

With the demonstration key

A demonstration key shows everything about an extract's structure and writes one record per segment. While an extract opened with it is open, a gold banner under the menu bar says so: Demo — one record per segment. Everything else works, but nothing you write here is an export. It cannot be closed.

• The records of a segment show the first record only — the one a demo write puts into the table. The line under the records reads Demo - one record per segment, of the … this one holds instead of a count. The list of fields is complete.
• The write dialogs repeat the limit above Start. In Write to database, Compare check sums is greyed out: the extract's totals cover every record, and a demo writes one. Your choice for it is kept for when you write with a real key.
• The result of a demo write that went through reads Complete — for a demo run, in amber rather than green. Every table or file was created and each holds one record; the byte checksum and the check sums are listed as not run, because the extract's figures cover all records. A demo database has every table, every column and every type of a real one, so a table with one row in it does not look wrong. Do not hand it to an audit.

Seeing what is in the extract

The Segments pane lists every segment of the extract: its name, the SAP application it belongs to, its description and the number of records the extract holds for it. The descriptions are in the language of the extract. The count at the top of the pane is the number of segments.

• Search name or description narrows the list to the segments whose name or description contains what you type.
• Name and Records sort the list; click again to reverse the order.
• Click a segment to see its fields underneath it: their position, name and description. Click it again to close them.

Narrowing the list down by application

Beside each segment name stands the SAP application it belongs to, as the extract states it: FI, SD, MM and so on. The box Application above the list shows only the segments of one application, and its list says how many segments each entry leaves. Two entries are not applications:

• Cross-application — the segments the extract marks with *, master and customising tables such as TXW_CLIENT and TXW_CNTRY that belong to every application rather than to one. Their chip in the list shows the *; hold the pointer over it to see the word.
• No application — the segments for which the extract names none. These are usually DaRT's own structures, such as TXW_META and TXW_CHKSUM. The entry is offered only when the extract has such segments.

The counts are taken from the segments the extract holds, so they add up to the figure under All.

Looking at the records of a segment

Click the number of records beside a segment. A window opens with the first 1,000 records of that segment, one column per field, so that you can see what the data looks like before writing it anywhere.

• The line under the table says how many records are shown of how many the segment holds. The records shown are the first ones in the extract, in the order SAP wrote them — not a sample.
• Click a column heading to sort by it. The line above the table says what the rows are sorted by; the order applies to the rows shown, not to the whole segment.
• Fields and Records at the top switch between the records and the list of the segment's fields, with their position and description.

Several of these windows can be open at once, for example to compare two segments.

Writing the extract

Tick the segments you want to write in the Segments pane. Select all shown ticks every segment the search and the application filter leave; Clear unticks them all. The line beside the write buttons says how many segments are selected.

Then click Write to database or Write to files in the middle pane, or choose them from the Run menu. Both stay greyed out until at least one segment is ticked; while nothing is ticked, the line beside the buttons reads Select at least one segment to write.

Writing to a database

Write to database opens a window with two tabs. Database and checks says where the extract goes and what is checked afterwards; Advanced says where the tables go inside the database and what they are called.

Choose the database:

• SQLite
• H2
• PostgreSQL
• SQL Server
• DuckDB
• MS Access

For databases that create a single file on the file system (SQLite, H2, DuckDB, MS Access), you have the option to select a file on the filesystem. Use New file… to say where a database that does not exist yet goes, and Existing file… to add the tables to a database that is already there. Writing into an existing database adds tables to it; it does not replace the file. As soon as the file is chosen, the window says which tables it already holds.

For these databases, you can alternatively, and for the other databases you must, enter a connection string, and where the database asks for one, a user and a password. For the databases for which you must enter a connection string, it is understood that there is a running server process that DaRT Reader can connect to. The password is not written anywhere — not into the settings and not into the log.

Please see the documentation of the respective JDBC drivers or database system on how to form the connection string:

Database	Documentation
SQLite	https://sqlite.org/c3ref/open.html
H2	https://www.h2database.com/html/features.html#database_url
PostgreSQL	https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING
SQL Server	https://learn.microsoft.com/en-us/sql/connect/jdbc/building-the-connection-url?view=sql-server-ver17
DuckDB	https://duckdb.org/docs/stable/clients/java
MS Access	https://ucanaccess.sourceforge.net/site.html#examples

ℹ️ Note: If you enter a connection string, leave out the protocol prefix. For example, in jdbc:postgresql://localhost:5432/mydb, leave out jdbc:postgresql:.

ℹ️ Note: Writing to SQL Server is considerably faster if you append ;useBulkCopyForBatchInsert=true to the connection string you enter here. The driver then sends each batch through its bulk copy path instead of as an ordinary insert; nothing else about the run changes. Measured against SQL Server 2022 in a container, one million rows over four connections: about 23,000 rows per second without it and between 105,000 and 154,000 with it — one machine, one server, so take it as an order of magnitude.

The option is the driver's own and brings related settings whose defaults differ from those of a plain insert. See the save command for what to check before switching it on.

The column types used for amounts and quantities differ between these databases, and SQLite has a limit worth knowing about before you pick it. See Amounts and quantities.

Under Check what was written you choose which comparisons run after the write — see Checking what was written. Your choice is kept for the next run, like a setting.

On the Advanced tab:

• Schema puts every table into a schema of the database. The schema has to exist already; DaRT Reader creates the tables, never the schema. Its name is matched exactly as you type it, upper and lower case included — many databases change the case of a name when it is created without quotes, so give it as the database shows it. SQLite and MS Access have no schemas; leave the field empty for them. See The schema is not there.
• Name prefix becomes part of every table's name, for example peter_ makes peter_TXW_FI_POS.
• Rename gives a segment, or one of its fields, a different name in the database.
• What will be created lists every table under the name it will have, schema and prefix included. A name the database already holds is refused before anything is written.

Click Start. The window shows each segment with the records written so far. Pause holds the write and keeps its connections; Resume carries on. Closing the window stops the write and asks first.

Writing CSV files

Write to files opens a window for the folder the files go into. Click Choose… or type the path. The window lists what is already in the folder and which files will be created, one per segment, named <segment-name>.csv.

If the folder already holds a file of one of those names, nothing is written and nothing is overwritten; the window names every file in the way, so that they can all be moved at once. See A file of that name is already in the folder.

Click Start. Each file exists from the start, and its size tells you how far it is. Pause holds the export; closing the window stops it and asks first — the files written by then stay where they are.

The two characters the files are written with, and whether they start with a byte order mark, are settings: see CSV files under Changing the settings.

The result

When a write ends, the window shows its result in three boxes, one under the other: the verdict, the list of segments or files, and the checks.

Write again goes back to the form, with the target looked into anew. Open the log opens the log narrowed to this run, and Copy this report to clipboard copies the result as text.

A database write is also kept under Run > Results, so that its result can be opened again after the window is closed.

Making a box of the result taller or shorter

Point at the gap between two boxes until the pointer becomes a double arrow, then drag it up or down. The boxes above and below the list change height, and the list takes up the difference. A box made shorter than its content scrolls. It cannot be made shorter than its heading and one line. When you resize the window, the list takes the difference. The heights are not kept: a result reopened from Run > Results, or a new one after Write again, starts at the usual heights again.

Checking what was written

The progress counts records. It tells you that rows arrived, not what is in them: a value written into the wrong column, cut short by the column type, or lost converting the extract's text into a number would count just the same. Two checks on the Write to database window look at the values themselves. They are independent — you can switch on either, both or neither — and they answer different questions.

Compare every value reads every value of every row back out of the database and folds it into a running total for its column. What is compared at the end is one total per column, one over whole rows, and the number of rows — a handful of numbers per segment rather than a comparison per cell. Every cell is still covered, and a changed value changes its column's total; what the check cannot do is say which value. It names the segment and the column. It changes nothing about the database, but it does take time: every value has to be read back. On a test extract of 287 segments and 99 016 records written to a local SQLite file, the write takes about 1.1 seconds and the comparison another 0.9. Against a database that is slower to write to, the share is smaller. This is the check that finds a single damaged value among millions.

Compare sampled rows takes 1 000 records of each segment — Records per segment changes the number — and compares them record by record, so that a difference can be named down to the row and to the byte in the extract the record came from. It finds what is systematic — a mapping that goes wrong for a whole column, where a hundred records are already enough to be sure — and it is not the check that finds one damaged row in a million.

To find a row again it needs to know which record it came from, so it adds two columns, DR_VOL_ID and DR_SRC_OFFSET, to every table it writes. It is on unless you turn it off, so this is worth knowing before you hand the database on: anyone you give it to — an auditor included — sees two columns that are not in the extract. The two columns are part of the tables, so the decision is made when the tables are created.

Compare check sums compares the totals the extract states in TXW_CHKSUM against the sums of the written tables — the one check that holds the database against something SAP computed rather than against what DaRT Reader made of the extract. It needs TXW_CHKSUM in the database and writes it as well, even if that segment is not ticked; the line at the foot of the window counts it among the tables.

None of the checks has any effect on writing CSV files. The files are meant to be the extract, and the two columns have no place in them.

If a check finds a difference, the run is reported as failed and the message names what differs — even though every record arrived. That is deliberate. What the run is for is a database that holds what the extract holds, and a difference means it does not, however many rows are in the tables.

Changing the settings

Choose File > Settings…. The settings are grouped; the column on the right explains the field you are in, with what it is set to by default, what it may be, and when a change takes effect.

Setting	Description
Reading and writing
Segments read at once	How many readers take the extract apart at the same time. A reader holds one payload file open, and a single large table is shared among several readers. 8 by default.
Number of connections	The number of database connections. This caps how many tables are filled at the same time. Does not apply to MS Access, which is always written through one connection — see An MS Access write stops moving.
Chunk size per connection	The number of records written to the database in one batch.
Connection pool
Connection timeout	How long, in milliseconds, a caller waits for a free connection before the run gives up.
Max lifetime	How long, in milliseconds, a connection is kept before it is retired and replaced.
Validation timeout	How long, in milliseconds, the pool may spend checking that a connection is still alive.
CSV files
Field separator	The character written between two fields of a CSV file. A German Excel expects ; and opens a comma separated file as a single column. Empty follows this machine's region.
Decimal separator	The character written inside an amount in a CSV file. With the wrong one the file opens in columns but the amounts arrive as text and cannot be summed. Empty follows this machine's region.
Byte order mark	Three bytes at the head of each CSV file that tell Excel the file is UTF-8. Without them a file opened by double-click on Windows is read in the ANSI code page and umlauts arrive wrong. On unless you turn it off.
Licence
Licence key	The license key you received during onboarding.
Diagnostics
Log level	How much detail is written to the log file. warn by default; raise it to debug when support asks.

A value the setting cannot take is refused under its field as you type, and Save stays greyed out until it is corrected; see A settings value is refused. Reset beside a field puts its default back. Nothing is written until you click Save.

The checks after a database write are not settings any more: they are chosen on the Write to database window, and the choice is kept from one run to the next.

Where the settings are stored

The settings, including the license key, are kept per user and outside the installation directory, so they survive an update and do not need administrator rights:

Platform	File
Windows	%APPDATA%\taxforge\DaRT Reader\settings.json
macOS	~/Library/Application Support/com.taxforge.dartreader/settings.json

Each user of a machine therefore has their own settings. To keep them somewhere else, start the application with -Ddart-reader.config-dir=<directory>.

Log files are written next to them, under %LOCALAPPDATA%\DaRT Reader on Windows and ~/Library/Logs/DaRT Reader on macOS, where Console.app picks them up.

The log level can also be set for a single run through the DART_READER_LOG_LEVEL environment variable, which takes effect a moment earlier than the setting and so captures the first few lines written during startup as well.

You can tradeoff higher throughput (size divided by time) with more memory consumption if you increase the Number of connections or the Chunk size per connection parameter. With number-connections = 5 and chunk-size-per-connection = 5000, batches of 5000 records will be written in 5 parallel connections amounting to 25,000 records. If the throughput is too high, you might get Out of memory errors. If this happens, you have to increase the memory of the JVM.

Both of those describe the writing side. Segments read at once describes the reading side, and it applies whether you write to a database or to files. Reading and parsing the extract is usually the slower half of a run, so raising this shortens the run noticeably.

It counts readers rather than segments: a reader holds one payload file open, and a single large table is shared among several readers — whether DaRT wrote it into one payload file or spread it over six.

Increasing memory of the JVM

From the installation directory, follow the relative path app on Windows and Contents/app on Mac OS and open the file DaRT Reader.cfg. At the end of this file in a section called [JavaOptions], make the following additions:

java-options=-Xms512m
java-options=-Xmx4g

This adds 512mb initial heap size and 4g maximum heap size. Increase these parameters if the JVM process runs out of memory.

Viewing the log

Choose View > Log. The result of a write has a link of its own, Open the log, which opens the log already narrowed to that run.

The window and the command line beye each write their own log files, and each rolls over at 5 MB, keeping three older files and deleting the oldest. The log window reads all of them and shows them as one log, oldest at the top, newest at the bottom — you do not have to know which file a line is in. A stack trace stays together with the message it belongs to.

Show narrows the log in time: The run that just finished (when opened from a result), Today, Everything, or From … to … for whole days of your choosing.

Level and From narrow it by severity and by program. Each chip counts what it would show in the chosen time, so a level with nothing in it shows 0 rather than seeming not to work. A chip switched off hides its lines; the count beside the chips says how many of how many lines are shown.

Two notes appear above the list when they apply:

• This log is written at level warn … — the log only holds what the configured log level lets through. With the default warn, there are no info or debug lines to find, and an empty level means never recorded, not nothing happened. Support will usually ask you to raise the level to debug and run again.
• This log does not go back to … — once the oldest file has been deleted, the log cannot answer for the time before its oldest entry. A range that reaches back that far is not empty at its start, it is unanswerable. A line across the list marks where the log begins.

Copy puts the lines shown on the clipboard, for a message to support. Show in Finder (Show in Explorer on Windows) opens the folder with the log files. Read again reads them anew, for a log that is still being written to.

Contacting support

Help > About DaRT Reader shows what support will ask for first:

• Licence: the license key with all but its last four characters hidden, and what the licensing service said about it. The license is checked when an extract is opened and again when it is written. Before the first of those, the status reads not asked yet, which does not mean that something is wrong.
• This installation: the version, the Java runtime, the operating system, and your region with the CSV separators that follow from it. Bundled means the runtime that came with the installer.
• Where things are: the folder holding your settings and the folder holding the log. Paths are written the way you can paste them into Finder's Go to Folder or Explorer's address bar. This is the folder actually in use, so a start with -Ddart-reader.config-dir shows that folder. Show in Finder (Show in Explorer on Windows) opens the folder, and Open the log opens the log window.

Copy details puts all of this on the clipboard as text, ready to paste into a message to support. The license key is copied hidden, as it is shown.