Input · Folders and loops

S3 Directory

Lists the objects in an S3 location that match a pattern, one row per object, so a loop can process each one.

Also known as: Batch of S3 files

How it works

Add the source from Add data source → S3 with a glob URI (* or **, for example s3://bucket/prefix/*): a glob always lists its objects, while a URI naming one object reads it as a table. The node lists the objects matching Object glob, for example s3://bucket/prefix/**/*.parquet; blank lists the URI the source was configured with. Each row carries uri, key and name. Wire it into a Loop's items port and, inside the loop, bind an S3 Reader's Object URI to the uri column.

Inputs and outputs

Settings

Example

All of last year's daily files under one prefix: Object glob s3://data-lake/events/2025/**/*.parquet. The loop runs once per object and collects the rows into one table.

More in Folders and loops

• Directory — Lists the files in a folder, one row per file, instead of reading their rows. Wire it into a loop to process every file in turn.
• DB Directory — Lists the tables in a database connection, one row per table, so a loop can run once per table. You can filter the list by name.
• Sheets — Lists the sheets of an Excel workbook, one row per sheet, so a loop can read each sheet in turn.
• Current item — Sits inside every loop and gives you the item the loop is on right now, such as the current file read as a table. It is added for you and can't be removed.
• File — Reads the file whose path sits in a column of its input: Parquet, CSV, JSON, Excel or a map file, on disk or in S3. Wire a loop's Current item into it to read a different file each time.