crawling
One of my favourite features of Fediverse is how different instances (whether it be Mastodon, Lemmy or Pixelfed) cooperate with each other over using the ActivityPub protocol. This is a fundamental feature of decentralised social media. For instance (pun unintended), users of mastodon.social can communicate with mastodonapp.uk, and
API
An essential skill for any web scraper or data scientist is to know how to collect information from a publicly available REST API. In short, a REST API is a very simple web service where simple HTTP requests (just like a web browser) are used to collect data usually in
pandas
EDIT (Feb 2023): As of 09/02/2023, Twitter is closing down public access to the API and replacing it with a paid service. In a separate blog post, we covered the basics for learning how to stream tweets from Twitter using Python. This is ideal if you are interested