Home

GLAMpipe is an extensible open source web-app for transforming datasets and uploading data and media to online repositories.

GLAMpipe can be used to:

• browse metadata
• importing metadata + files
• process metadata + files
• exporting metadata + files

GLAMpipe can import from:

• csv files (local/online)
• DSpace 6
• Internet Archive
• Wikidata
• Map warper
• Finna
• Plone (initial version)
• ePrints (very initial version)

GLAMpipe can export to:

• csv
• DSpace 6
• Plone (initial version)
• Omeka S
• Wikimedia Commons

GLAMpipe is a wrapper for other open-source programs:

• pdftotext
• CLD2 (Compact Language Detector)

• GROBID (requires separate install)

  The basic idea of GLAMpipe is to read data and files from different sources, process the metadata and export the data and files to different destinations. The workflow is based on documents and nodes instead of rows and cells.

  A building block of the data flow is called a node. Each node can do one thing. For example, a node can import data from a source. Another node can modify the data in different ways: split text, combine strings, format Wikimedia templates etc. Finally, a node can export data as files or as data in different formats and to different services like Wikimedia Commons or Wikidata.

  If you want to start experimenting directly, go to the demo. If you plan to work more extensively with the tool, check out installation pages.