# Introducing tidygraph

## July 7, 2017

I’m very pleased to announce that my new package `tidygraph`

is now
available on CRAN. As the name
suggests, `tidygraph`

is an entry into the tidyverse that provides a tidy
framework for all things relational (networks/graphs, trees, etc.). `tidygraph`

is a relatively big package in terms of exported functions (280 exported
symbols) so all functions will not be covered in this release note. I will
however provide an overview of all the areas that `tidygraph`

touches upon so you
should have a pretty good grasp on what the package can do for you.

## Tidy network data?

There’s a discrepancy between relational data and the tidy data idea, in that relational data cannot in any meaningful way be encoded as a single tidy data frame. On the other hand, both node and edge data by itself fits very well within the tidy concept as each node and edge is, in a sense, a single observation. Thus, a close approximation of tidyness for relational data is two tidy data frames, one describing the node data and one describing the edge data.

### The tbl_graph object

Underneath the hood of `tidygraph`

lies the well-oiled machinery of igraph,
ensuring efficient graph manipulation. Rather than keeping the node and edge
data in a list and creating `igraph`

objects on the fly when needed, `tidygraph`

subclasses `igraph`

with the `tbl_graph`

class and simply exposes it in a tidy
manner. This ensures that all your beloved algorithms that expects `igraph`

objects still works with `tbl_graph`

objects. Further, `tidygraph`

is very
careful not to override any of `igraph`

s exports so the two packages can coexist
quite happily.

To underline the tidyness of the `tbl_graph`

class the print method shows the
object as two tibbles along with additional network information.

`tbl_graph`

objects can be created directly using the `tbl_graph()`

function
that takes a node data.frame and an edge data.frame. On top of that, `tidygraph`

also provides coercion from a huge amount of relational data structures. The
following list gives the packages/classes that can currently be converted to
`tbl_graph`

s, using the `as_tbl_graph`

function:

`data.frame`

,`list`

,`matrix`

from`base`

`igraph`

from`igraph`

`network`

from`network`

`dendrogram`

and`hclust`

from`stats`

`Node`

from`data.tree`

`phylo`

and`evonet`

from`ape`

`graphNEL`

,`graphAM`

,`graphBAM`

from`graph`

(in Bioconductor)

For all of the coercions you can expect that data on the nodes and edges are
kept and available after conversion to `tbl_graph`

:

Lastly, `tidygraph`

also wraps the multitude of graph constructors available in
`igraph`

and exports them under the `create_*()`

family of functions for
deterministic constructors (e.g. the call to `create_ring(10)`

above) and the
`play_*()`

family for constructors that incorporate sampling (e.g.
`play_erdos_renyi()`

for creating graphs with a fixed edge probability). All of
these functions provide a consistent argument naming scheme to make them easier
to use and understand.

### Meet a new verb…

There are many ways a multitable setup could fit into the tidyverse. There could
be an added qualifier to the verbs such as `mutate_nodes()`

and `filter_edges()`

or each verb could take an additional argument specifying what is targeted e.g.
`arrange(..., target = 'nodes')`

. Both of these approachable are viable but
would require a huge amount of typing as well as being taxing to support down
the line.

The approach used by `tidygraph`

is to let the data object itself carry around
a pointer to the active data frame that should be the target of manipulation.
This pointer is changed using the `activate()`

verb, which, on top of changing
which part of the data is being worked on, also changes the print output to show
the currently active data on top:

As can be seen, `activate()`

takes a single argument specifying the part of the
data that should be targeted for subsequent operations as an unquoted symbol.
`tidygraph`

continues the naming conventions from `ggraph`

using `nodes`

and
`edges`

to denote the entities and their connections respectively, but
`vertices`

and `links`

are allowed synonyms inside `activate()`

.

The current active data can always be extracted as a tibble using `as_tibble()`

## The dplyr verbs

Using `activate()`

it is possible to use the well known `dplyr`

verbs as one
would expect without much hassle:

In the above the `.N()`

function is used to gain access to the node data while
manipulating the edge data. Similarly `.E()`

will give you the edge data and
`.G()`

will give you the `tbl_graph`

object itself.

Some verbs have effects outside of the currently active data.
`filter()`

/`slice()`

on node data will remove the edges terminating at the
removed nodes and `arrange()`

on nodes will change the indexes of the `to`

and
`from`

column in the edge data.

While one might expect all of `dplyr`

s verbs to be supported in that manner,
there is a clear limitation in the relational data structure that requires rows
to maintain their identity. Thus, `summarise()`

and `do()`

are not allowed as
there is no clear interpretation of how alterations on the node and edge data
with these verbs should be interpreted. If these operations are required I
suggest applying them to a tibble representation and then joining the result
back in.

Speaking of joining, all joins from `dplyr`

are supported. Nodes and edges are
added and removed as required by the join. New edge data to be joined in must
have a `to`

and `from`

column referencing valid nodes in the existing graph.

### Expanding the vocabulary

On top of what has been showed so far, `tidygraph`

provides an assortment of
graph specific verbs that can be used to power your analysis and manipulation.
Analogous to `bind_rows()`

, `tidygraph`

provides three functions to expand your
data: `bind_nodes()`

and `bind_edges()`

append nodes and edges to the graph
respectively. As with the join functions `bind_edges()`

must contain valid
`from`

and `to`

columns. `bind_graphs()`

allows you to combine multiple graphs
in the same graph structure resulting in each original graph to become a
component in the returned graph.

While `bind_graphs()`

cannot be used to create edges between the merged graphs
`graph_join()`

can do just that. It merges nodes using a `full_join()`

semantic
and keeps the individual edges from both graphs:

The standard `dplyr`

verbs protects the `to`

and `from`

columns in the edge data
in order to avoid accidental modification of the graph topology. If changing of
the terminal nodes are necessary the `reroute()`

verb will come in handy:

As can be seen, reroute works pretty much as a `to`

and `from`

specific
`mutate()`

, with the added benefit of incorporating a subset operator if only a
few edges should be changed.

## Making the most of graphs

While being able to use the `dplyr`

verbs on relational data is nice and all,
one of the reasons we are dealing with graph data in the first place is because
we need some graph-based algorithms for solving our problem at hand. If we need
to break out of the tidy workflow every time this was needed we wouldn’t have
gained much. Because of this `tidygraph`

has wrapped more or less all of
`igraph`

s algorithms in different ways, ensuring a consistent syntax as well as
output that fits into the tidy workflow. In the following we’re going to take a
look at these.

Central to all of these functions is that they know about which graph is being
computed on (in the same way that `n()`

knows about which tibble is currently in
scope). Furthermore they always return results matching the node or edge
position so they can be used directly in `mutate()`

calls.

### Node and edge types

On the top of our list of things we might be interested to know about is whether
nodes or edges are of specific types such as *leaf*, *sink*, *loop*,
etc. All of these functions return a logical vector indicating if the node or
edge belong to the specified group. To easily find functions that queries types,
all functions are prefixed with `node_is_*`

/`edge_is_*`

.

Another example could be to remove loop edges using `filter(!edge_is_loop())`

.

### Centrality

One of the simplest concepts when computing graph based values is that of
*centrality*, i.e. how central is a node or edge in the graph. As this
definition is inherently vague, a lot of different centrality scores exists that
all treat the concept of *central* a bit different. One of the famous ones is
the pagerank algorithm that was powering Google Search in the beginning.
`tidygraph`

currently has 11 different centrality measures and all of these are
prefixed with `centrality_*`

for easy discoverability. All of them returns a
numeric vector matching the nodes (or edges in the case of
`centrality_edge_betweenness()`

).

It is quite difficult to a priori decide which centrality measure makes most sense for a problem at hand so having easy access to a large range of them in a common syntax is a boon.

### Clustering

Another common operation is to group nodes based on the graph topology,
sometimes referred to as *community detection* based on its commonality in social
network analysis. All clustering algorithms from `igraph`

is available in
`tidygraph`

using the `group_*`

prefix. All of these functions return an integer
vector with nodes (or edges) sharing the same integer being grouped together.

### Node pairs

Some statistics are a measure between two nodes, such as distance or similarity
between nodes. In a tidy context one of the ends must always be the node defined
by the row, while the other can be any other node. All of the node pair
functions are prefixed with `node_*`

and ends with `_from`

/`_to`

if the measure
is not symmetric and `_with`

if it is; e.g. there’s both a `node_max_flow_to()`

and `node_max_flow_from()`

function while only a single `node_cocitation_with()`

function. The other part of the node pair can be specified as an integer vector
that will get recycled if needed, or a logical vector which will get recycled
and converted to indexes with `which()`

. This means that output from node type
functions can be used directly in the calls, e.g.

### Searches

An integral type of operation on graphs is to perform a search, that is, start
from one node and then traverse the edges until all nodes has been visited. The
most common approaches are either *breath first search* where all neighbors of
a node is visited before moving on to the next node, or *depth first serch*
where you move along to the next node immediately and only backtracks and visit
other neighbors when you’ve hit a dead end. Different statistics from these
searches are available in `tidygraph`

through the `bfs_*()`

and `dfs_*()`

family
of functions e.g. the distance to the start node along the search can be
obtained with `bfs_dist()`

/`dfs_dist()`

. The root node can be specified in the
same way as with node pairs. Sorting based on a search from the node with
highest centrality can thus be done with:

### Local measures

Often we find ourselves interested in the local neighborhood of a node for
various reasons. We might want to know the average degree around a node or the
number of triangles each node participate in. The `local_*()`

family of
functions provide access to a range of node measures that are dependent on the
local neighborhood of each node.

### All the rest

While an ontology of graph operations has been attempted in the different
functions above, there are some that falls outside. These have been lumped
together under the `node_*()`

and `edge_*()`

umbrellas and include things such
as topological ordering and Burt’s constraint among others. All of these
functions ensures a mutate-compatible output.

### Graph measures

Along with computations on the individual nodes and edges it is sometimes
necessary to get summary statistics on the graph itself. These can be simple
measures such as the number of nodes and edges as well as more involved measures
such as assortativity (the propensity of similar nodes to be connected). All of
these measures can be calculated through the `graph_*()`

function family and
they will all return a scalar.

## Mapping over nodes

Just to spice it all up a bit `tidygraph`

pulls `purrr`

into the mix and
provides some additional graph-centric takes on the familiar `map*()`

. More
specifically `tidygraph`

provides functionality to apply a function over nodes
as a breath or depth first search is carried out, while getting access to the
result of the computations coming before, as well as mapping over the local
neighborhood of each node. All of these function returns a list in their
bare bone form, but as with `purrr`

versions exists that ensures the output is of
a certain type (e.g. `map_bfs_dbl()`

).

### Mapping over searches

The search maps comes in two flavors. Either the nodes are mapped in the order
of the search, or they are mapped in the reverse order. In the first version,
each call will have access to the statistics and map results of all the nodes
that lies between itself and the root. In the second version each call will
have access to the results and statistics of all its offspring. Furthermore the
mapping function is passed the graph itself as well as all the search statistics
of the node currently being mapped over. An example would be to propagate the
*species* value in our iris clustering upwards as long as theirs agreement
between the children. For this to work, we will need the reverse version of a
breath first search to make sure that all children have been evaluated prior to
mapping over a node:

### Mapping over neighborhoods

The neighborhood map is exposed through `map_local()`

as well as its type safe
versions. The mapping function has a much simpler format as it simply gets
passed a subgraph representing the local neighborhood as well as the index of
the node in the original graph being mapped over. E.g. to get the number of
edges in the local neighborhood around each node, one would simply do:

## One last thing…

While the functions discussed above makes it easy to make slight changes to your
network topology it is less straightforward to make radical changes. Even more
so if the radical changes are only needed temporarily for the sake of a few
computations. This is where the new `morph()`

verb comes in handy (along with
the accompanying `unmorph()`

and `crystallise()`

verbs). In essence, `morph()`

lets you set up a temporary alternative version of your graph, make computations
on it using the standard `dplyr`

verbs, and then merge the changes back in
using `unmorph()`

. The types of alternative representations are varied and can
be extended by the user. Nodes can be converted to edges and the other way
around, both nodes and edges can be combined, and the alternate representation
does not need to cover the full original graph. Instead of trying to describe it
in words, let’s see how it plays out in use:

As can be seen, the morph syntax both handles multiple graphs, collapsed nodes
and changing edges to nodes, without any change in the mental model of the
operations. All morphing functions are prefixed with `to_*`

for easy discovery
and includes minimum spanning trees, complement graph, dominator tree etc. In
the case where you are interested to continue working with the morphed
representation as a proper `tbl_graph`

you can use the `cystallise()`

verbs that
removes any link to the original graph and returns a tibble with a row per graph
in the morphed representation (as a morph can result in multiple graphs):

## Wrapping it all up

I hope I have given you a small glimpse of what `tidygraph`

is all about. If
working with network data in the past has felt intimidating and strange
`tidygraph`

might feel more at home, but even if you’re a seasoned pro within
network analysis the package should provide a powerful but streamlined interface
to many operations.

### Roadmap

The next goal of my quest to revamp relational data analysis in R will be to
“rebuild” `ggraph`

around `tidygraph`

. This is not to say that the two are
incompatible at the moment — there’s full support through `ggraph`

s support for
`igraph`

— rather I want to only support `tbl_graph`

in the future as all
relevant data structures can be converted to this common format through
`as_tbl_graph()`

.

For `tidygraph`

itself, I have some more ideas I want to explore. Currently
missing from the whole package is any notion of modelling and it will be
interesting to see how this can fit in. Further, I have this wild idea about
providing a `tidygraph`

link to graph databases such as Neo4J in the same way
as `dbplyr`

provides an interface to SQL databases. Lastly, the current focus
has been on supporting the algorithms provided by `igraph`

. While an extensive
package, `igraph`

does not implement everything and there might be stuff lacking
that should be added down the line.

Take care…