Call to Action: Measuring Postdoc Impact

Opinion, Scientific Workforce

The views expressed here are my own and do not necessarily represent those of North Carolina State University, the National Postdoctoral Association, or the Graduate Career Consortium. 

The week of September 21-25, 2020 is National Postdoc Appreciation Week, a time for institutions and supervisors to express their appreciation for the contributions postdoctoral scholars make in a variety of areas from research to teaching and mentoring. 

As a quick refresher, a postdoctoral scholar (postdoc) is an individual with an advanced degree (Ph.D.) engaged in a period of additional training to build the skills and experiences necessary to launch themselves into an independent career of their choosing.

While traditionally a postdoc was considered the necessary "next step" in one's training progression to ultimately obtain a faculty position, postdocs move into a variety of careers after their time in postdoctoral training.

​The goal of postdoctoral offices are to support these scholars in their career and professional development. However, many postdoc offices are under-resourced to provide this much needed support to a critical component of the scientific workforce. Many are offices of 1-2 people responsible for supporting anywhere from 300 to 1,200+ postdocs. I believe one of the missing pieces allowing postdoc offices to obtain additional support is to better measure and report on the contributions postdocs make to the academic and research institutions that employ them. 

Postdocs' Contributions to Their Institutions

Postdoctoral scholars contribute much to the scientific enterprise and the research and teaching missions of universities.

Despite the varied and important roles they play, postdocs are often overlooked as a key constituency by administrators and leaders at research and academic institutions in the United States and globally.

I believe that measuring postdoctoral impact on the teaching/training and research/scholarship endeavors of institutions will better illuminate the key role they play in research organizations.
​I also hope this process leads to increased institutional support for the postdoc population.

What metrics would help quantify postdoctoral impact?

​The first task is to identify metrics that would be useful to gauge the impact postdocs have on their field, lab, peers, students, and university. Some metrics could include:

1. Papers published (# of papers published, # of 1st-authored papers published)
2. Students mentored (# of students mentored, dissertation/thesis supervision of graduate/undergraduate students)
3. Teaching (# hours lectured, # courses taught, # of students taught, course evaluations)
4. Grant money (% of contribution to advisor’s research grant applications, Co-Principal Investigator (PI) roles on grants, postdoc fellowships received)
5. Contribution to lab and project management (assisting their PI supervisors in managing experiments/projects/timelines, freeing up PIs to focus on other tasks) 
6. Patents filed, intellectual property developed
7. Service performed for the university (sitting on various committees, serving in a leadership role for a campus organization like a postdoctoral association)

Challenges in measuring this impact​

Some of these proposed metrics of postdoc contributions are difficult to quantify and, clearly, everything cannot be measured in a quantitative sense. However, we must try to develop systems to collect what information we have on postdoc impact. 

​Their are also structural challenges & barriers to measuring postdoc impact in these domains. 

1. Publications are often not published until 6-12 months after the first manuscript submission. Thus, a metric of publications will show postdoc impact in the past and not current impact. Conference presentations and preprint publications by postdocs may be additional metrics one could collect to gauge short-term scholarly impact. 
2. Postdoc mentoring of students is often not formalized (or even officially allowed). While a postdoc may contribute substantially to helping a graduate student in their dissertation work, the Principal Investigator (PI) of the lab (the postdoc's supervisor) is the mentor of record.
3. Regarding teaching, postdocs are often not allowed to be “instructor of record” on courses (except in the case of the small number of teaching postdocs, whose job duties include primarily teaching) and are often discouraged from teaching in place of conducting research work. However, lecturing and other teaching roles postdocs perform should be measured as this contributes to the overall educational mission of universities.
4. In the area of grants, postdocs often contribute substantially to grant applications, whether that be in the form of preliminary data they collect/analyze or assisting PIs in conceiving and/or writing of grant proposals. However, a postdoc cannot serve as a PI (and sometimes, depending on the policy of the training institution, Co-PI) on grant applications as they are not “faculty.” Thus, quantifying postdocs’ contribution to grant money received is complicated. There is no doubt, however, that they play a role (and sometimes substantial one) in many grant applications.
5. Measuring postdoc contributions to lab management is complex. It is the PI’s role to manage their labs. Thus, even if postdocs contribute substantially in assisting them in this role, some PIs may not feel that they can acknowledge this work as it could diminish their perceived role as the lab’s director.

Structural/Institutional Barriers to Measuring Postdoc Contributions 

​Many of the challenges of measuring and acknowledging postdoctoral contributions come down to the hierarchy of academia. Postdocs are “under” PIs whose own metrics of success largely mirror that of the postdoc. While this could suggest areas where there should be mutual benefit (both postdoc and supervisor/PI benefit from publications), there could also be cases where this situation introduces conflict. For example, a PI needs to secure external funding in their name to receive tenure at most research intensive academic institutions. Postdocs also need to show the ability (or potential) to secure external funding to transition into independent, tenure-track faculty positions. If postdocs cannot receive credit for their contributions to their supervisor’s grant application(s), however, one cannot truly measure their impact in this regard. This situation can also harm postdocs’ ability to secure independent faculty positions of their own, stifling their career progression.

One immediate way to address this and that I acknowledge is occurring with many PIs is to include mention of postdoc contributions to successful grant applications in letters of reference the PIs write on their behalf. In this way, a PI can make clear that the postdoc contributed substantially to a successful research grant application and will be capable of serving as PI on their own grants if they transition to an independent faculty position of their own. 

Importance of Measuring Postdoc Impact

​As the measurement of graduate student and postdoc career outcomes and progression has gained attention, so too should the measurement of postdocs’ contributions in their current roles. Standardization of the metrics used to assess postdoc contributions and impact need to be undertaken to make data robust and comparable across departments and institutions. In addition, a standardized collection method would allow data to be aggregated to quantify postdoc impact at a national or international level. 

How Postdoc Offices Can Benefit from Measuring Postdoc Impact

​From conversations had at the National Postdoctoral Association (NPA) annual meeting and with colleagues, postdoc offices are staffed relatively sparsely. It is often difficult for these offices to quantitatively support the value they bring to an institution. Thus, resource allocation to these offices are often minimal as the value proposition for their existence has not been made. While clearly postdocs’ receive the majority of their research training from their lab supervisors, the postdoc office at most institutions provides valuable support in assisting postdocs in their career & professional development, especially in regards to the pursuit of nonacademic careers. 

Thus, quantifying postdoc impact will allow postdoc offices to better advocate for additional resources to support this important (but often overlooked) population. ​

Final Thoughts

​While much progress has been made in improving the working conditions and support (stipends) for postdocs in the US driven by many dedicated individuals and the efforts of the NPA, a need to move toward a more structured, data-driven approach to demonstrating their impact is needed. Only by showing the impact postdocs make can the offices who support them effectively advocate for additional institutional resources to enhance the lives and training of postdocs. By doing so, we can more effectively nurture the next generation of researchers and allow them to leverage their skills, interests, and values to improving society in the variety of careers they pursue after their postdoctoral training. 

Publications on Postdoc Impact
Bringing the lab back in: Personnel composition and scientific output at the MIT Department of Biology

Contribution of postdoctoral fellows to fast-moving and competitive scientific research

Postdocs as Mentors - When it comes to everyday mentoring and training in the sciences, postdocs are the new PIs.

Useful tools to quantify publication metrics
Pubmetric - Authorship crawler that document the publishing record for trainees and their PIs via Pubmed

Publons - Track your publications, citation metrics, peer reviews, and journal editing work in a single, easy-to-maintain profile

​SciVal - From Elsevier. Allows you to visualize your research performance, benchmark relative to peers, develop strategic partnerships, identify and analyze new, emerging research trends, and create uniquely tailored reports. 

​See also
​NC State Libraries Research Impact & Metrics Landing Page 

Academic Analytics

NPA Institutional Policy Report & Database - a great model that currently focuses on postdoc offices reporting on postdoc benefits, salaries, and numbers in addition to office size and budgets ​