Evolving Scholarship in a Digital Age: Metrics and Management

Data Management

Data-driven research has become increasingly common lately, which has left researchers with questions about what to do with all their data. Federal grants in particular have requirements for making data accessible, and other funders and groups have similar rules in place. Even if access to data is not mandated by the terms of a grant or institution, a researcher may still want to consider sharing their data. As files and formats change over time, ensuring the long-term survival of research data is also a concern.

Creating a Data Management Plan
The first step to long-term data management is to create a data management plan. A librarian may be able to help you with this. You'll need to think about your data in order to determine the best format and method for storing it. For example, what is the context of your data? What format is it in? How could it be used in the future? Is there sensitive/confidential information in your data? There are many other considerations in creating a data management plan as well - all of them can inform you and anyone working with you on how best to keep your data. If you'd like to read more on data management plans or see examples, check out DMPTool.

Storing Research Data
Once you have a plan, you'll have a better idea of where and how to store your data. At Ithaca College, our Digital Commons is able to host data sets. Data on the Digital Commons is search-engine optimized, shared with the larger Digital Commons community, and easily accessible. The Digital Commons also tracks usage, so you will be able to tell how often people view your data. You can see an example data set on Digital Commons to see if that would work for you. If you prefer to go with another option, there are many available online. Sites like re3data.org can help you find other data repositories.

Preparing Research Data

If you plan to make your data open, either by choice or due to a funding mandate, you may want to take a few steps to ensure your data is usable and understandable to others. Prepping your data doesn't mean altering it - you should always be sharing your raw data. However, there are some things you can do to increase its value. You can make a data set more usable by:
  • Using non-proprietary formats (.csv instead of Excel, for example)
  • Using clear names for files, tabs, and columns/rows
  • Keeping data and metadata terminology consistent throughout the data set (and using standardized terminology where possible)
  • Cleaning up notes that might be unclear to someone unfamiliar with your work
  • Eliminating or explaining any stray figures, notes, or calculations
For more on making data usable, check out "Some simple guidelines for effective data management" (Borer et al, 2009). 

Altmetrics

Altmetrics are non-traditional, Social Web-based measures of the reach and influence of scholarly output. Some examples of altmetrics are downloads, bookmarks, blog posts, shares, and citation indexes. Altmetrics data can be used to explain the amount and type of online attention that items receive. According to the authors of Meaningful Metrics, these measures show the impact of a wider type of items beyond traditional research journals (software, data sets, and slide decks, for example) and can show areas of influence outside of disciplinary boundaries and formal academic communities (though citations in public policy documents and references in patents, learning objects, and press coverage, for instance).

Several publishers include altmetric data for their journals, including the American Chemical Society,  PLoS, and Wiley.  Open access archives (such as ArXiv) and Institutional Research Repositories also now contain this type of article level metrics.  An Altmetric Bookmarklet (once installed) is available to capture this data in Google Scholar.

Using Altmetrics for Evaluation

When used with traditional, citation-based bibliometrics, altmetrics can form a more complete and detailed picture of the impact of scholarship. They can sometimes indicate the potential impact of research on society or a field of study. Altmetrics can be especially useful in discplines where journal articles are not the primary output.

Facuty can use altmetrics to
  • Determine who, where, and how their research is being used
  • Quantify how their research is being used
  • Spot potential collaborators in their field
  • Show the influence of their work inside and outside their field of study and academia
  • Learn what others think of their research
  • Demonstrate successful outreach
  • Pinpoint and filter other sources of importance
  • Provide a context for their work
  • Solicit research funding
  • Identify publications to submit to

For Tenure and Promotion

The Leiden Manifesto for Research Metrics

Guide to Preparing a Dossier for Promotion or Tenure
University of Colorado Denver Medical School

Guidelines for Preparing and Reviewing Promotion and Tenure Dossiers 2017-18  
Indiana University-Purdue University Indianapolis

Guidelines for the Evaluation of Digital Scholarship in Art and Architectural History
College Art Association and the Society of Architectural Historians 

Guidelines for Evaluating Work in Digital Humanities and Digital Media
Modern Language Association

In a CV
Recommendations for including altmetrics in a CV include providing contextual information, such as percentiles, maps, and qualitative data. Here's one example from the CV of Trevor A. Branch, Biology, University of Washington:






Additional examples can be found at What Are Altmetrics? and in the University of Maryland Bibliometrics and Altmetrics: Mesuring the Impact of Knowledge guide.

Contact Us

picture of Calida Barboza

Calida Barboza

Electronic Resources Librarian
(607) 274-1892
picture of Abby Juda

Abby Juda

Natural Sciences Librarian
(607) 274-3889

Finding Data Sets

There are many, many sources of open data on the internet, but finding the type and level of data set you're looking for can be a challenge. Here are some suggestions of places to start:

DataOne
DataOne is a data repository containing biological and environmental data sets. DataOne is easy to search and use.

Data.gov
Data.gov is a good place to go for data from the federal government. Contains lots of public data, although it's not always in the most usable format.

DataDryad
DataDryad provides free data sets and other educational resources. Primarily biology data.

Institutional respositories 
Many colleges and universities host data sets in their institutional repositories. It's a little more work, but looking in some IRs might lead to great finds. You can check out Michigan's Deep Blue Repository to see an example. 

Ithaca College Library guide to Statistics and Data Sets

Data Management Tools

Creating a Data Management Plan

DMP Tool 

Data Management Plan Resources and Examples

Guidelines for Effective Data Management Plans

Prepping your Data

Open Refine (formerly Google Refine)
An open source tool for data transformation and cleanup.

Nesstar Publisher
Data and metadata conversion tools to prepare your data for publication.

Analyzing your Data

R
Free statistical analysis program - with a bit of a learning curve. 

SPSS
Statistical analysis program available on library computers.

Data Visualization

Tableau Public
Free data visualization software. Allows you to create interactive, embedd-able graphics.

Raw
A fairly simple web-based application for creating and customizing data visualizations.

Exhibit
Open source software to create interactive, data-rich websites. Best for location-related datasets.

Altmetrics Tools

Impactstory 
Draws from a variety of social and scholarly data sources, including Facebook, Twitter, CiteULike, PubMed, Scopus, CrossRef, scienceseeker, Mendeley, Wikipedia, slideshare, Dryad, and figshare. Offers free widget that can be embedded into repositories.

PLOS Article Level Metrics 
Tracks item-level views, saves, citations, recommendations, and discussions of scholarly output.

Altmetric.com
Tracks social media sites, newspapers, and magazines. Altmetrics is based on three main factors: the number of individual mentioning a paper, where the mentions occurred (e.g. newspaper, a tweet), and how often the author of each mention talks about scholarly articles. Altmetric has been adopted by Springer, Nature Publishing Group, Scopus, and BioMed Central, among others. Altmetrics offers a free bookmarklet that can be added to the bookmarks toolbar and used to get altmetrics on articles with Digital Object Identifiers (DOIs) or identifiers in open access databases such as PubMed Central or arXiv. Altmetrics will only work on Chrome, Firefox or Safari.

Kudos
Provides a unique one-stop shop for multiple metrics relating to your publications: page views, citations, full text downloads and altmetrics. 

PLOS ALMs (Article Level Metrics)
Custom searches to track the access and reuse of articles published in PLOS journals.

Figshare  
Allows researchers to publish all of their data in a citable, searchable and sharable manner. All data is persistently stored online under the most liberal Creative Commons licence, waiving copyright where possible. This allows scientists to access and share the information from anywhere in the world with minimal friction