Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ jobs:
- name: Generate libdoc
run: |
mkdir -p docs
libdoc src/Tables docs/keywords.html
libdoc -F MARKDOWN src/Tables docs/keywords.html
cp docs/keywords.html docs/index.html
touch docs/.nojekyll

Expand Down
12 changes: 9 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,10 @@ dev = [
"pytest-cov",
"pre-commit",
"mypy",
"robotframework-robocop"
"robotframework-robocop",
"robotframework==7.5",
"markdown",
"Pygments"
]

[project.urls]
Expand Down Expand Up @@ -114,13 +117,16 @@ exclude_lines = [
dependencies = [
"ruff",
"pytest",
"pytest-cov"
"pytest-cov",
"robotframework==7.5",
"markdown",
"Pygments"
]
[tool.hatch.envs.dev.scripts]
lint = "ruff check ."
atest = "robot tests/atest/"
utest = "pytest --cov=Tables --cov-report=term-missing --cov-branch --cov-fail-under=95"
docs = "libdoc src/Tables results/keyword_docs.html"
docs = "libdoc -F MARKDOWN src/Tables results/keyword_docs.html"

[tool.mypy]
python_version = "3.10"
Expand Down
248 changes: 133 additions & 115 deletions src/Tables/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,24 +31,24 @@ class Tables(HybridCore):
"""
``TableLibrary`` is a generic automation library for working with files like csv, parquet, etc.

== Table of content ==
## Table of content

%TOC%

== Supported File Types ==
# Supported File Types
- CSV
- Excel
- Parquet
- TXT -> will be interpreted as CSV file
- More coming soon...

== Supported File Encoding ==
# Supported File Encoding
- utf-8
- utf-16
- latin-1
- further more... (define your own file encoding with the given library argument)

== Live Data Streaming ==
# Live Data Streaming
Some keywords allow you to write the inserted or appended data directly into an output table file.
By default, the table library writes the modified table data first into the internal cache.
In this case, you decide when to write the data into a file.
Expand All @@ -58,108 +58,128 @@ class Tables(HybridCore):
Afterwards, every keyword that allows the on-the-fly streaming, writes the new data directly into the file system instead of modifying the table just in the cache.

Example:
| # enable the streaming on-the-fly
| Configure Streaming True Test
|
| VAR ${filepath} = ${CURDIR}/results/test_writer_live_streaming_append_row.csv
| VAR @{headers} = counter random
| ${uuid} = Tables.Create Table headers=${headers} file_path=${filepath}
|
| # append ten rows on-the-fly within the given csv file
| FOR ${i} IN RANGE ${10}
| ${random} = Generate Random String 10
| VAR @{row} = ${i} ${random}
| Tables.Append Row ${row}
|
| ${table_content} = Tables.Read Table ${filepath}
| Should Be Equal ${table_content}[-1][1] ${random}
| END

== Excel Files ==
```robotframework
*** Test Cases ***
Enable live data streaming
# enable the streaming on-the-fly
Configure Streaming True Test

VAR ${filepath} = ${CURDIR}/results/test_writer_live_streaming_append_row.csv
VAR @{headers} = counter random
${uuid} = Tables.Create Table headers=${headers} file_path=${filepath}

# append ten rows on-the-fly within the given csv file
FOR ${i} IN RANGE ${10}
${random} = Generate Random String 10
VAR @{row} = ${i} ${random}
Tables.Append Row ${row}

${table_content} = Tables.Read Table ${filepath}
Should Be Equal ${table_content}[-1][1] ${random}
END
```

# Excel Files

We have included a basic handling of ``Excel`` files,
but for more complex excel features, please take a look at the following library:
[https://pypi.org/project/robotframework-excelsage|robotframework-excelsage].
\nThis library got especially written to work with Excel files, sheets, etc...

== Examples ==
=== File Format - CSV ===
| # Reading CSV file with header column
| ${content} = Tables.Read Table ${CURDIR}${/}testdata${/}example_01.csv
| ${result} = BuiltIn.Evaluate "${content}[0][0]" == "index"
| BuiltIn.Should Be True ${result}
|
|
| # Reading CSV file without header column
| Tables.Configure Ignore Header True
| ${content} = Tables.Read Table ${CURDIR}${/}testdata${/}example_01.csv
| ${result} = BuiltIn.Evaluate "index" not in "${content}"
| BuiltIn.Should Be True ${result}

=== File Format - Parquet ===
| Tables.Configure File Type Parquet
| ${content} = Tables.Read Table ${CURDIR}${/}testdata${/}example_05.parquet
| ${result} = BuiltIn.Evaluate "${content}[0][0]" == "_time"
| BuiltIn.Should Be True ${result}

=== Create new empty table - don't save to file system ===
| # Create some data which should be inserted into the new table
| VAR @{headers} = name age
| VAR @{person1} = Michael 34
| VAR @{person2} = John 19
|
| # Create empty table object - internally in cache
| ${uuid} = Tables.Create Table headers=${headers}
|
| # Append some rows
| Tables.Append Row ${person1}
| Tables.Append Row ${person2}
| Count Table ${uuid} Rows equal ${3}
|
| # Append a column
| VAR @{column1} = city MG ERL
| Tables.Append Column ${column1}
| Count Table ${uuid} Columns equal ${3}
|
| # Optional: Set new table cell value
| Get Table Cell 1 1 equals 34
| Tables.Set Table Cell 25 0 1
| Get Table Cell 1 1 equals 25
|
| # Insert a new row into the existing table object
| VAR @{insert_row} = Lu 26 Hamburg
| Insert Row ${insert_row} 0
| Get Table Cell 1 0 equals Lu
| Count Table ${uuid} Rows equal ${4}

=== Create new empty table - save to file system ===
| # Generate new headers which should be used in the table
| VAR @{headers} = name age
|
| # Create new table object
| ${uuid} = Create Table ${headers}
|
| # Generate some random data & append as rows to new table
| FOR ${_} IN RANGE ${100}
| ${a} = Generate Random String
| ${b} = Generate Random String
| VAR @{data} ${a} ${b}
| Tables.Append Row ${data}
| END
|
| # Ensure that data got written into internal table object
| Count Table ${uuid} Rows equals ${101}
|
| # Write table to specific file path -> write from cache into persistant file
| Write Table ${uuid} ${CURDIR}/results/test_writer_new_table.csv
|
| # Check table content again, but now read table from file path!
| Count Table ${CURDIR}/results/test_writer_new_table.csv Rows equals ${101}

=== Parquet ===
| Tables.Configure File Type Parquet
| ${content} = Tables.Read Table ${CURDIR}${/}testdata${/}example_05.parquet
| ${result} = BuiltIn.Evaluate "${content}[0][0]" == "_time"
| BuiltIn.Should Be True ${result}
[robotframework-excelsage](https://pypi.org/project/robotframework-excelsage).

This library got especially written to work with Excel files, sheets, etc...

# Examples

## File Format - CSV

```robotframework
*** Test Cases ***
Reading CSV file with header column
${content} = Tables.Read Table ${CURDIR}${/}testdata${/}example_01.csv
${result} = BuiltIn.Evaluate "${content}[0][0]" == "index"
BuiltIn.Should Be True ${result}


Reading CSV file without header column
Tables.Configure Ignore Header True
${content} = Tables.Read Table ${CURDIR}${/}testdata${/}example_01.csv
${result} = BuiltIn.Evaluate "index" not in "${content}"
BuiltIn.Should Be True ${result}
```

## File Format - Parquet

```robotframework
*** Test Cases ***
Read Parquet Table File
Tables.Configure File Type Parquet
${content} = Tables.Read Table ${CURDIR}${/}testdata${/}example_05.parquet
${result} = BuiltIn.Evaluate "${content}[0][0]" == "_time"
BuiltIn.Should Be True ${result}
```

## Create new empty table - don't save to file system

```robotframework
*** Test Cases ***
Create empty table saved in cache
# Create some data which should be inserted into the new table
VAR @{headers} = name age
VAR @{person1} = Michael 34
VAR @{person2} = John 19

# Create empty table object - internally in cache
${uuid} = Tables.Create Table headers=${headers}

# Append some rows
Tables.Append Row ${person1}
Tables.Append Row ${person2}
Count Table ${uuid} Rows equal ${3}

# Append a column
VAR @{column1} = city MG ERL
Tables.Append Column ${column1}
Count Table ${uuid} Columns equal ${3}

# Optional: Set new table cell value
Get Table Cell 1 1 equals 34
Tables.Set Table Cell 25 0 1
Get Table Cell 1 1 equals 25

# Insert a new row into the existing table object
VAR @{insert_row} = Lu 26 Hamburg
Insert Row ${insert_row} 0
Get Table Cell 1 0 equals Lu
Count Table ${uuid} Rows equal ${4}
```

## Create new empty table - save to file system

```robotframework
*** Test Cases ***
Create empty table & save to file system
# Generate new headers which should be used in the table
VAR @{headers} = name age

# Create new table object
${uuid} = Create Table ${headers}

# Generate some random data & append as rows to new table
FOR ${_} IN RANGE ${100}
${a} = Generate Random String
${b} = Generate Random String
VAR @{data} ${a} ${b}
Tables.Append Row ${data}
END

# Ensure that data got written into internal table object
Count Table ${uuid} Rows equals ${101}

# Write table to specific file path -> write from cache into persistant file
Write Table ${uuid} ${CURDIR}/results/test_writer_new_table.csv

# Check table content again, but now read table from file path!
Count Table ${CURDIR}/results/test_writer_new_table.csv Rows equals ${101}
```
"""

def __init__( # noqa: PLR0913
Expand All @@ -175,17 +195,15 @@ def __init__( # noqa: PLR0913
missing_as_none: bool = False,
):
"""
``TableLibrary`` can be controlled by the following arguments:

| =`Argument`= | =`Description`= |
| ``file_type`` | Choose the file type to test initially. |
| ``file_encoding`` | Defiine the file encoding - select from given hints or define your own by string value. |
| ``separator`` | Define a separator for parsing the files. |
| ``ignore_header`` | Define if headers in files should be ignored. Default is ``False`` |
| ``line_terminator`` | Define the required line terminator for your table files. Default is ``False`` |
| ``quoting`` | Define which values should be surrounded with quotes, please check the CSV quoting for more details. Default is ``MINIMAL`` |
| ``quoting_character`` | Define quoting character to use for writing table files. Default is ``\"`` |
| ``missing_as_none`` | Return parsed missing values as Python ``None`` instead of pandas missing values. Default is ``False`` |
Args:
file_type: Choose the file type to test initially.
file_encoding: Defiine the file encoding - select from given hints or define your own by string value.
separator: Define a separator for parsing the files.
ignore_header: Define if headers in files should be ignored. Default is ``False``
line_terminator: Define the required line terminator for your table files. Default is ``False``
quoting: Define which values should be surrounded with quotes, please check the CSV quoting for more details. Default is ``MINIMAL``
quoting_character: Define quoting character to use for writing table files. Default is ``\"``
missing_as_none: Return parsed missing values as Python ``None`` instead of pandas missing values. Default is ``False``
"""

# required variables for SettingsScope mechanism
Expand Down
Loading
Loading