Addressing what we know about the changes, what we can currently do with the VOSON Lab tools and future developments.
Recently, social media companies have announced major changes to API access and data, specifically for Twitter and Reddit. In this post we cover what we have found so far, and the implications in the use of the VOSON lab suit of tools: vosonSML, VOSONDash and voson.tcn, as at July 2023.
Reddit announced changes to its API effective on 30 June 2023. This publication summarises the changes to the API. Basically, Reddit is restricting API access to commercial entities that require large-scale data usage. Reddit data can still be accessed for free for academic purposes (non-profit), as long as it is below the published data-usage threshold.
As of July 1, 2023, Reddit enforced two different rate limits for the free access tier:
100 queries per minute per OAuth client id (i.e. those using OAuth for authentication)
10 queries per minute, for those who are not using OAuth for authentication.
As at July 2023, it is possible to access Reddit data using vosonSML (v. 0.37.7) and VOSONDash (0.5.11). The Reddit API end-point used does not require OAuth authentication for vosonSML, which means that there should not be problems with small collections.
There is currently a default wait time of 3-5 seconds between collection requests. Users can change this default behaviour by modifying the WaitTime parameter in the vosonSML Collect() function, as documented here.
Given Reddit’s statement that there is a maximum of 10 API requests per minute (when OAuth authentication is not used), then users should set the WaitTime parameter to a minimum of 6 seconds so as to avoid hitting the rate limit. However as above, we expect the current version of vosonSML will be fine for small Reddit collections, and therefore, VOSON Dashboard can be used for small collections, with the current default wait time. We will be updating the default wait time in a future version of vosonSML.
For Twitter, changes to the API access have been significant. This post by rtweet developers summarises the changes of API access since February until June 2023. On 9 February 2023, Twitter cut off free access to both of its APIs (v2 and v1.1) and included a paid basic tier. Legacy Twitter API access tiers, such as Standard (v1.1), Essential (v2), Elevated (v2), and Premium were deprecated by June 2023. Any applications and projects that a user had not switched over to one of the new tiers were automatically placed in the default new Free access tier, which supports V1.1 Media Endpoints, V1.1 OAuth endpoints (Login with Twitter), V2 Manage Tweet endpoints
and V2 users/me endpoint, for write only use cases.
Based on the information above, if you have a developer account, with access to the v.1.1 API endpoint, you should be able to collect Twitter data with vosonSML using a paid tier (note: as of July 2023 we in the VOSON Lab have not tested collection using a paid tier).
Also, please note that vosonSML only accesses the Twitter v1.1 API via rtweet and does not support the newer v2 API. You can refer to the voson.tcn package if you are interested in using the v2 API to collect and analyse Twitter conversation networks.
As of July 2023, there is still a lot of uncertainty around Twitter data collection, and in particular, what will happen to the previous Twitter access for academics.
We are keeping a close eye on developments with Twitter, or X(!?), but we are also extending the frontiers of VOSON by looking into other data sources such as Mastodon.
As of July 2023, there have been no changes to the YouTube API and tne VOSON tools can collect YouTube comment networks.
Our collection of WWW hyperlink networks is, of course, unaffected by any changes to social media platforms.
Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
For attribution, please cite this work as
Ackland, et al. (2023, July 21). VOSON Lab Code Blog: Changes to APIs: Mapping the implications in data collection using the VOSON Lab suite of tools. Retrieved from https://vosonlab.github.io/posts/2023-07-21-voson-and-changes-to-apis/
BibTeX citation
@misc{ackland2023changes,
author = {Ackland, Robert and V., Francisca Borquez and Gertzel, Bryan},
title = {VOSON Lab Code Blog: Changes to APIs: Mapping the implications in data collection using the VOSON Lab suite of tools},
url = {https://vosonlab.github.io/posts/2023-07-21-voson-and-changes-to-apis/},
year = {2023}
}