Twarc-Cloud supports collecting twitter data from Twitter’s API using Twarc running in AWS. It is designed to be super scalable and cheap and not require a server or a sysadmin.

Twarc-Cloud supports filter streams, user timelines, and searches.

Twarc-Cloud documentation

Contents:

• Requirements
  • Python 3 and pip 3
  • Terraform
  • Twitter API developer account and app
  • AWS account
  • Twarc-Cloud
  • Honeybadger (optional)
• Quick start
  • Setup
  • Create a user timeline collection
  • Download the collection
  • Stop the collection
• Collection types
  • User timeline
  • Search
  • Filter stream
• Twarc-Cloud commandline
  • General
  • Collection configuration commands
  • Collection commands
  • Harvest commands
• Twitter API keys
  • Application keys
  • User keys
  • Managing keys
• Administration
  • Unlocking a collection
  • Removing AWS environment
  • Logs
• Design and implementation details
  • Design principles
  • AWS
  • Harvester implementation

Acknowledgements

Twarc-Cloud is inspired by and borrows heavily from DocNow’s Twarc and George Washington University Libraries’ Social Feed Manager.

Indices and tables

• Index
• Module Index
• Search Page