Extract Structured Data

Extract Structured Data

Type: page-extract-table  ·  Package: Browser v1.0.10  ·  Output: DataTable

Extracts table-shaped data from the page and turns it into a DataTable, ready to loop over, filter or write to Excel. It works with HTML tables (<table>) and also with data that looks like a table without being one: calendars, card grids or layouts built with CSS grid/flex. If the data is paginated, it can go through all the pages for you.

It’s far more efficient than reading cell by cell with Get Text: one activity brings you the whole table.

How it works

It has two modes, chosen with mode:

Mode	What for	Parameters it uses
table (default)	An HTML <table>	selector
smart	Any visual grid, without <table>	container, cells, headers, rowHeaders

If you leave mode empty but set cells, smart mode is used. In both modes, the activity first waits for the root element (the table or the container) to be visible, up to timeout; if the selector matches several, it uses the first one.

table mode

It locates the <table> by its selector; if the selector points to a container, it uses the first table inside it.

• Headers: the <th>/<td> cells of the <thead>. If the table has no <thead>, the first row is taken as the header.
• Rows: the rows of the <tbody> (or, without <thead>, every row after the first). Only the <td> cells of each row are read; rows without any <td> (subtitles made only of <th>) are skipped.
• Text of each cell: its visible text (innerText), with no leading or trailing spaces.

smart mode

It rebuilds the grid from the on-screen position of each element:

1. Inside container (or the whole page if you leave it empty) it looks for the elements of cells, headers and rowHeaders.
2. Columns: each headers element defines a column, ordered left to right; its text is the column name. Without headers, columns are inferred by grouping cells that sit in the same vertical band.
3. Rows: each rowHeaders element defines a row, ordered top to bottom, and its text goes into an extra first column. Without rowHeaders, rows are inferred by grouping cells that sit in the same horizontal band.
4. Each cell goes to the column and row whose band contains its center (or to the nearest one). If two cells land in the same position, their texts are joined with |; positions without a cell stay empty.

Each element’s text is taken with spaces and line breaks collapsed into a single space. Elements must be rendered on the page: anything hidden has no useful position.

Set selectors in smart mode

cells, headers and rowHeaders must match all the repeated elements, not one in particular. If you capture one of them with UI Discover, its fallback CSS is used (tag and classes, e.g. div.event), which covers all its siblings. If you write them by hand, use CSS (.event, td.hour): text= and xpath= aren’t supported in these three parameters.

Pagination

If you set nextPage (the “next” button), after reading the first page it repeats up to maxPages pages in total:

1. It stops if the button isn’t visible or is disabled: a disabled attribute, aria-disabled="true", or a disabled, is-disabled, btn-disabled, page-item-disabled or pagination-disabled class on the button (or disabled / is-disabled on its parent element).
2. It clicks the button and waits for the complete new page. Before the click it takes a fingerprint of the whole content: in table mode, the text of every body row; in smart mode, the text and position of every cells element inside the container. After the click it waits, up to timeout, for that fingerprint to change and stay still for a moment (300 ms; 1 s if the new page comes back empty), for the network to go idle (up to 3 s) and for the requests started by the click to finish (up to 5 s). This way it doesn’t read a half-rendered page or a “Loading…” state, and it takes no longer than needed even when two pages start with the same row.
3. It reads the page and appends its rows. It stops if a page comes back without rows.

If the content doesn’t change within timeout after the click, the activity fails instead of reading the same page again: duplicate rows are never appended and incomplete data is never returned silently. It usually means nextPage points to an element that doesn’t load the next page, or that the site leaves the button active on its last page without disabling it; in that case limit the run with maxPages. If the content changes but keeps changing (for example, a clock inside the table), when timeout expires it is read as it is and a warning is logged. Since version 1.0.10.

Headers are taken from the first page only.

Required parameters

None. (In table mode you need selector; in smart mode, cells.)

Optional parameters

Parameter	Type	Default	Description
page	ZoanPageSession	—	The page session. Omit it inside a body; outside, pass the session variable (e.g. page = page).
mode	string	"table"	Extraction mode: table (reads a <table>) · smart (rebuilds the grid by position). Empty with cells set: smart.
selector	selector	—	table mode: the <table> element (or a container that holds it). Required in that mode.
container	selector	whole page	smart mode: the container that scopes the extraction (the root of the grid or calendar).
cells	selector	—	smart mode: the data cells. Each element is placed in the grid by its position. Required in that mode.
headers	selector	—	smart mode: the column headers. Their text names the columns and their horizontal position defines each column.
rowHeaders	selector	—	smart mode: the row labels (the left axis, e.g. the hours). Adds a leading column with the label and aligns each row by its vertical position.
nextPage	selector	—	“Next page” button. If set, it extracts all the pages until the button disappears or is disabled. If a click doesn’t change the content, the activity fails (see Pagination).
maxPages	int	100	Maximum number of pages to extract, counting the first.
timeout	number	30000	Maximum time in ms to wait for the table or container, and for it to change after each page. Empty, 0 or negative: the project settings value or, if none, 30000.

Output

Returns a DataTable in which all columns are text (string):

Mode	Columns	Rows
table	One per header cell, with its text. An empty header is named Col1, Col2… by its position. If a row has more cells than headers, ColN columns are added.	One per row with data; missing cells stay empty.
smart	With rowHeaders: first the label column (Col1), then one per header. Columns without header text (or all of them, if there are no headers) are named Column 1, Column 2…	One per row label (or per inferred band).

Edge cases:

• A table with headers but no rows returns a DataTable with its columns and 0 rows.
• If the selector doesn’t point to any table, or in smart mode no cell matches, the activity fails (see below).

Example

Extract a paginated list of transactions and loop over it:

Extract Structured Data   selector = (#tablaMovimientos)   nextPage = (.btn-siguiente)   maxPages = 10   → output: movimientos
Log   message = "Rows extracted: " + rowCount(movimientos)

For Each   items = movimientos   itemVariable = fila
 └─ activities:
      Log   message = str(fila["Date"]) + " — " + str(fila["Amount"])

Read a weekly meeting-room booking agenda that doesn’t use <table>: the days are the columns and the hours, the rows:

Extract Structured Data   mode = smart   container = (.agenda)   cells = (.booking)   headers = (.agenda-day)   rowHeaders = (.agenda-hour)   → output: agenda
For Each   items = agenda   itemVariable = fila
 └─ activities:
      Log   message = str(fila["Col1"]) + " Monday: " + str(fila["Monday"])

After extracting

A DataTable integrates with the rest of the platform: loop over it with For Each, access cells with str(fila["Column"]), or write it straight to Excel/CSV. See DataTable.

Common errors

Message	Cause and fix
A table selector is required in 'table' mode.	You’re in table mode (or mode is empty and there’s no cells) and didn’t set selector. Set it, or switch to mode = smart.
No table found at the given selector.	The element exists but isn’t a <table> and doesn’t contain one. Point to the table, or use smart mode if the data isn’t in a <table>.
No elements matched 'cells' inside the container.	In smart mode, the cells CSS found nothing inside container. Check the selector (it must cover all the cells) and that the container is the right one.
Pagination: after clicking the next-page button on page N, the content did not change within … ms (… rows extracted so far). …	The click on nextPage didn’t change the content within timeout. Check that the selector points to the button that loads the next page; if the site is slow, increase timeout; if the last page leaves the button active, limit with maxPages. Since version 1.0.10.
Timeout 30000ms exceeded	The table or container didn’t become visible in time. Check the selector or wait first with Wait For.
A column named '…' already belongs to this DataTable.	Two headers have the same text (case-insensitive). Use smart mode with other headers, or extract with another selector.
'page' must be a ZoanPageSession.	page is empty outside a body.

Related activities

• Get Text — read a single value instead of a whole table.
• Scroll — for “infinite scroll” lists (not paginated with a button).
• For Each — loop over the extracted rows.