VOSON Lab Code Blog: Egocentric Networks from Twitter timelines

Bryan Gertzel

Updated for rtweet v1.0.2 and vosonSML v0.32.7.

This article does not require Twitter API keys, but does require a twitter account and for the user to authorise the rtweet rstats2twitter app when prompted.

Introduction

Egocentric networks or ego nets are networks that focus on a particular actor (the ego) and map out their connections to other actors. In an ego net other actors are referred to as alters, and by collecting the outward expanding connections of alters, ego nets of varying degrees can be constructed (see Hogan, 2011, p. 168). Some literature and software refer to these as neighborhood networks, with varying orders instead of degrees. For example in a friendship network, a neighborhood network of the first order (1.0 degree) contains just the friends of the ego, whereas a network of the second order (sometimes second step) also contains “friends of friends” (a 2.0 degree ego net).

By collecting the tweets in a Twitter users timeline, and the timelines of users referenced, we can create a 1.0, 2.0 or 1.5 degree network for the ego. A 1.5 degree network is similar to the 1.0 degree, except it also contains relationships or ties between the alters, or “between friends” of the ego from the previous friendship network example.

It should be noted that by using user timelines that this is not necessarily a friendship network, but instead a network of twitter users who are associated through tweet activity. This kind of ego net can lead to insights beyond simply declared friendships (obtained from Twitter’s friend/follower metadata) as the network structure is the result of users interactions on the platform over a period of time.

This post will demonstrate how to construct an ego networks from a twitter timelines using the rtweet package to collect tweets and vosonSML to create an actor network.

Collect the ego timeline

The first step is to collect the ego’s timelime. In this post we will be using the VOSON Lab @vosonlab twitter account, and collecting the Twitter timeline using rtweet. The Twitter API restricts the number of timeline tweets that can be collected to the most recent 3,200 tweets, but we can set this to a lesser value e.g most recent 100 tweets, and also use the same parameter for alters timelines for the purposes of this demonstration.

library(dplyr)
library(DT)
library(rtweet)
library(vosonSML)

# get twitter user timeline
ego_tweets <- get_timeline(c("vosonlab"), n = 100, token = NULL)

# convert rtweet data into vosonSML format
ego_tweets <- ego_tweets |> ImportRtweet()

# create actor network from timeline tweets
ego_net <- ego_tweets |> Create("actor", verbose = TRUE)

Generating twitter actor network...
-------------------------
collected tweets | 100
tweet mention    | 62
tweet            | 25
retweet          | 66
quote mention    | 13
quote            | 9 
nodes            | 41
edges            | 175
-------------------------
Done.

A result of 41 nodes indicates that there are 40 alters in the network with ties to the ego.

Figure 1: vosonlab 1.0 degree actor network

Collect timelines of alters

From the previous step we created an actor network represented as nodes and edges dataframes from the ego’s tweet timeline. We can now use this to extract all of the user ids of the alters in the network.

Note that we have not specified the degree of the ego net at this stage, however by virtue of the twitter data (timeline tweets) having all been created by the ego user, we can assume all of the alters (referenced users) are connected to the ego in this network.

# get ego user id
ego_user_id <- ego_net$nodes |>
  filter(screen_name == "vosonlab") |> pull(user_id)

# get list of alter user ids from network
alter_user_ids <- ego_net$nodes |>
  filter(user_id != ego_user_id) |> distinct(user_id) |> pull()

Using the alters user ids the timeline tweets can be collected and imported into vosonSML as follows:

# get 100 most recent tweets from all of the alters timelines
# and convert to vosonSML format
alters_tweets <- alter_user_ids |>
  get_timeline(n = 100, retryonratelimit = TRUE) |>
  ImportRtweet()

# Error: Number of tweet observations does not match number of users. 3526 != 99

Please note there seems to be an inconsistency in timeline results for this version of rtweet and the following workaround can be used instead:

# workaround for rtweet timeline users issue
get_alters_timelines <- function(x) {
  ImportRtweet(get_timeline(user = x, n = 100, retryonratelimit = TRUE))
}

# collects timelines individually and place into a list
require(purrr)
alters_tweets <- map(alter_user_ids, get_alters_timelines)

Alternatively, if you have your own API access the vosonSML Collect function can also be used with the endpoint = "timeline" parameter:

# requires a previously saved vosonSML twitter auth object
auth_twitter <- readRDS("~/.vsml_auth_tw")

alters_tweets2 <- auth_twitter |>
  Collect(
    endpoint = "timeline",
    users = alter_user_ids,
    numTweets = 100,
    verbose = TRUE
  )

# Collecting timeline tweets for users...
# Requested 4000 tweets of 150000 in this search rate limit.
# Rate limit reset: 2022-08-22 06:21:30
# 
# tweet        | status_id           | created            
# --------------------------------------------------------
# Latest Obs   | 1560130378366562304 | 2022-08-18 05:04:02
# Earliest Obs | 1544727926645596162 | 2022-07-06 17:00:12
# Collected 3525 tweets.
# Done.

Create an actor network

Now that all of the tweets from the alters timelines have also been collected, the data can be merged and a single actor network created. This actor network can be considered a 2.0 degree network, as it contains not only the associations or “friends” from the ego’s timeline, but also the associations or “friends” of the alters from their timelines.

# combine all of the tweets from ego and alters timelines using vosonSML merge
tweets <- do.call(Merge, alters_tweets)
tweets <- Merge(ego_tweets, tweets)

# create actor network from combined timeline tweets
actor_net <- tweets |> Create("actor", verbose = TRUE)

Generating twitter actor network...
-------------------------
collected tweets | 3626
tweet mention    | 1030
tweet            | 1160
retweet          | 1657
reply mention    | 735
reply            | 584
quote mention    | 187
quote            | 232
nodes            | 1818
edges            | 5585
-------------------------
Done.

Here we can see an actor network of 1818 nodes and 5585 edges, substantially larger than our initial actor network.

Figure 2: vosonlab 2.0 degree actor network

Optionally add user metadata as node attributes

At this point we can optionally add some user metadata to our network as node attributes. This allows us to change visual properties of the network graph based on actor attributes. For example, we could map the node size to number of followers a twitter user may have.

Please note this step requires a vosonSML twitter auth object if you want to use the look up feature for complete users’ metadata.

# this step requires a previously saved vosonSMML twitter auth object
auth_twitter <- readRDS("~/.vsml_auth_tw")

# add user profile metadata
actor_net_meta <- actor_net |>
  AddUserData(tweets, lookupUsers = TRUE, twitterAuth = auth_twitter)

Here is a sample of the actor metadata available and an example of how it can be presented and explored using a data table:

# node attributes
names(actor_net_meta$nodes)

 [1] "user_id"                 "screen_name"            
 [3] "u.user_id"               "u.name"                 
 [5] "u.screen_name"           "u.location"             
 [7] "u.description"           "u.url"                  
 [9] "u.protected"             "u.followers_count"      
[11] "u.friends_count"         "u.listed_count"         
[13] "u.created_at"            "u.favourites_count"     
[15] "u.verified"              "u.statuses_count"       
[17] "u.profile_banner_url"    "u.default_profile"      
[19] "u.default_profile_image" "u.withheld_in_countries"
[21] "u.derived"               "u.withheld_scope"       
[23] "u.utc_offset"            "u.time_zone"            
[25] "u.geo_enabled"           "u.lang"                 
[27] "u.has_extended_profile"

# explore actors metadata
actors_table <- actor_net_meta$nodes |>
  filter(user_id %in% c(ego_user_id, alter_user_ids)) |>
  mutate(u.screen_name = paste0("@", screen_name)) |>
  select(name = u.screen_name,
         display = u.name,
         locationu = u.location,
         description = u.description,
         followers = u.followers_count,
         tweets = u.statuses_count) |>
  slice_head(n = 5)

library(reactable)

reactable(actors_table, bordered = TRUE, striped = TRUE, resizable = TRUE,
          wrap = FALSE, searchable = TRUE, paginationType = "simple")

Create an ego subgraph

A 1.5 degree network can be useful to reveal the associations between an ego’s alters. This can be achieved by creating a subgraph of the 2.0 ego network that retains only the previously identified alters (see below igraph::induced_subgraph). As we know every alter is connected to the ego so it is also often useful to visualise ego networks without the ego as it is then easier to observe clustering.

library(igraph)

# use the vosonSML to convert the network dataframes into an igraph object
g <- actor_net_meta |> Graph()

# create a subgraph with ego removed
g2 <- induced_subgraph(g, c(alter_user_ids))

g2

IGRAPH 953ba4f DN-- 40 1922 -- 
+ attr: type (g/c), name (v/c), screen_name (v/c), u.user_id
| (v/c), u.name (v/c), u.screen_name (v/c), u.location (v/c),
| u.description (v/c), u.url (v/c), u.protected (v/l),
| u.followers_count (v/n), u.friends_count (v/n),
| u.listed_count (v/n), u.created_at (v/c), u.favourites_count
| (v/n), u.verified (v/l), u.statuses_count (v/n),
| u.profile_banner_url (v/c), u.default_profile (v/l),
| u.default_profile_image (v/l), u.withheld_in_countries
| (v/x), u.derived (v/c), u.withheld_scope (v/l), u.utc_offset
| (v/l), u.time_zone (v/l), u.geo_enabled (v/l), u.lang (v/l),
| u.has_extended_profile (v/l), status_id (e/c), created_at
| (e/c), edge_type (e/c)
+ edges from 953ba4f (vertex names):

As we saw in our initial actor network constructed from only the ego’s timeline we now have 40 nodes again, matching the number of alters. This actor network has many more edges however, as 1922 ties or relations between the alters were captured from the collection of the alters timelines.

Visualise with an interactive visNetwork plot

Using the igraph and visNetwork package we can create a simplified and undirected ego network graph of alters. Community detection can be performed and visualised using the igraph walktrap clustering algorithm and Fruchterman-Reingold force-directed layout. We can further map some visual properties of nodes to attributes - with node size corresponding to the node degree, edge width to combined weight, and color to clustering community group.

library(visNetwork)

# combine and weight the edges between nodes
E(g2)$weight <- 1
g2 <- igraph::simplify(g2, edge.attr.comb = list(weight = "sum"))
g2 <- as.undirected(g2)

# perform some community detection using a random walk algorithm
c <- walktrap.community(g2)
V(g2)$group <- membership(c)

# map visual properties of graph to attributes
E(g2)$width <- ifelse(E(g2)$weight > 1, log(E(g2)$weight) + 1, 1.1)

V(g2)$size <- degree(g2) + 5
V(g2)$label <- paste0("@", V(g2)$u.screen_name)

visIgraph(g2, idToLabel = FALSE) |>
  visIgraphLayout(layout = "layout_with_fr") |>
  visOptions(
    nodesIdSelection = TRUE,
    highlightNearest = TRUE
  )

Figure 3: vosonlab 1.5 degree actor network

The final result is an ego net with some clear associations. Colours and placement of nodes found to represent some interesting domains and community relationships between the Twitter @vosonlab account and its timeline network alters. Isolates represent more distant connections with no detected community ties.

Hogan, B. (2011). Chapter 11 - visualizing and interpreting facebook networks [Book Section]. In D. L. Hansen, B. Shneiderman & M. A. Smith (Eds.), Analyzing social media networks with NodeXL (pp. 165–179). Morgan Kaufmann. https://doi.org/10.1016/B978-0-12-382229-1.00011-4

Egocentric Networks from Twitter timelines