Dr. Peter Organisciak

Associate Professor, University of Denver, Research Methods and Information Science

Professional Positions

Associate Professor
University of Denver . (September 2023 - present )
Assistant Professor
University of Denver . (September 2017 - August 2023 )
Postdoctoral Research Associate
HathiTrust Research Center . (2015 - 2017 )
Research Intern
Microsoft Research . (April 2013 - August 2013 )

Education

Ph.D., University of Illinois at Urbana-Champaign, 2015

Major: Library and Information Science

Dissertation: Design Problems in Crowdsourcing: Improving the Quality of Crowd-Based Data Collection

Committee: Michael Twidale, J. Stephen Downie, Miles Efron, Jaime Teevan

MA, University of Alberta, 2010

Major: Humanities Computing - Library and Information Studies

Thesis: Why Bother? Examining the motivations of users in large-scale crowd-powered online initiatives

Committee: Lisa Given, Geoffrey Rockwell, Stan Ruecker

BA, McMaster University, 2008

Major: Communication Studies and Multimedia

Awards

Teaching Appreciation Dinner, University of Denver, March 2020
Best Short Research Paper Winner, iSchools Conference, June 2019
Notable Paper, Conference on Human Computation and Crowdsourcing, 2014
Outstanding Contribution Award, Canadian Society for Digital Humanities, 2014
Best Paper, Association for Information Science and Technology, 2011
Best Student Paper, Canadian Society for Digital Humanities, 2011

Media

It's Out of Africa meets Pretty Woman! The Art and Science of Mixture Descriptions
Katy Waldman, Slate (Lexicon Valley: A Blog about Language), November 23, 2015
Online crowds can guess what you want to watch or buy.
Hal Hodson, New Scientist, 2989, October 2, 2014

Awarded Grants

Measuring Original Thinking in Elementary School: A Computational Psychometric Approach

Amount: $964,081

Funder: Institute of Education Sciences

Grant Number: R305A200519

Investigators: Selcuk Acar (University of North Texas, Principal), Dumas, D. G. (Co-Principal), Organisciak, P. (Co-Principal)

Timeframe: 2020-2024

Subgrant: DU sub-grant: $451,988

Collaborative Analysis Liaison Librarians (CALL)

Amount: $883,034

Funder: Institution of Library and Museum Services

Grant Number: RE-13-19-0027-19

Investigators: Wade Bishop (Principal, University of Tennessee Knoxville), Carole Tenopir (UTK, Co-Principal), Suzie Allard (UTK, Co-Principal), Organisciak, P. (Director)

Timeframe: 2019-2022

Subgrant: DU total: $129,746.00

Text Duplication and Similarity in Massive Digital Collections

Amount: $276,943

Funder: Institute of Museum and Library Services

Grant Number: LG-86-18-0061-18

Investigators: Organisciak, P. (Principal)

Timeframe: 2018-2021

Exploring the Billions of Words in the HathiTrust Corpus with Bookworm: HathiTrust + Bookworm

Amount: $324,841

Funder: National Endowment for the Humanities

Investigators: Downie, J. S. (Illinois, Principal), Aiden, E. L. (Rice University)

Timeframe: 2014-2017

Products

Research Websites and Tools

Open Creativity Scoring (OCS)

(2207 unique users, June 2020-August 2022)

Software

HTRC Features Reader

A mature, long-developed library for working with non-consumptive book data.

Open Scoring

Tools for creativity scoring.

Publications

Journal Articles

Research methods and the use of visual representation in library and information science research

Journal of the Association for Information Science and Technology

DOI:

Performance and Perception of Creativity and Academic Achievement in Elementary School Students: A Normal Mixture Modeling Study

Journal of Creative Behavior

DOI:

Measuring original thinking in elementary school: Development and validation of a computational psychometric approach

Journal of Educational Psychology

DOI:

Additional Info:

Automated Scoring of Figural Tests of Creativity with Computer Vision

Journal of Creative Behavior

DOI:

Additional Info:

Note: Acar and Organisciak contributed equally.

Lost in translation? Not for Large Language Models: Automated divergent thinking scoring performance translates to non-English contexts

Thinking Skills and Creativity

Volume: 50

DOI:

URL:

Automated feedback and creativity: On the role of metacognitive monitoring in divergent thinking

Psychology of Aesthetics, Creativity, and the Arts

DOI:

Beyond semantic distance: Automated scoring of divergent thinking greatly improves with large language models

Thinking Skills and Creativity

Volume: 49

DOI:

Additional Info:

How do the kids speak? Modeling child-directed language for educational use

Information and Learning Science

DOI:

Additional Info:

What makes children’s responses to creativity assessments difficult to judge reliably?

Journal of Creative Behavior

DOI:

The effects of research data management services: Associating the data curation life cycle with open research output

College & Research Libraries

DOI:

Approximate Nearest Neighbor for long document relationship labeling in digital libraries

International Journal of Digital Libraries

DOI:

Measuring Flexibility of Thinking Using Text-Mining

Frontiers in Psychology

URL:

Defensive Driving: CAR:POOL as a Model for Proactive Postsecondary Planning and Connection

Contemporary School Psychology

DOI:

Job analyses of Earth science data managers: A survey validation of competencies to inform curricula in research data management education

Journal of Education for Library and Information Science

DOI:

Improving Text Relationship Modeling with Artificial Data

Journal of Information Science

DOI:

Applying automated originality scoring to the verbal form of Torrance Tests of creative thinking

Gifted Child Quarterly

DOI:

Giving shape to large digital libraries through exploratory data analysis

Journal of the Association for Information Science and Technology

DOI:

Four text-mining methods for measuring elaboration

Journal of Creative Behavior

Volume: 55

DOI:

The psychology of professional and student actors: Creativity, personality, and motivation

PloS ONE

Volume: 15

DOI:

Is the reliability of objective originality scores confounded by elaboration?

Creativity Research Journal

DOI:

Measuring divergent thinking originality with human raters and text-mining models: A psychometric comparison of methods

Psychology of Aesthetics, Creativity, and the Arts

DOI:

The HathiTrust Digital Library's potential for musicology research

International Journal on Digital Libraries

DOI:

The MIREX grand challenge: A framework of holistic user-experience evaluation in music information retrieval

Journal of the Association for Information Science and Technology

Volume: 68

DOI:

Text mining in Python through the HTRC Feature Reader

The Programming Historian

Volume: 5

DOI:

A fragmentizing interface to a large corpus of digitized text: (Post)humanism and non-consumptive reading via features

Interdisciplinary Science Reviews

Volume: 40

DOI:

The design of an international social media event: A Day in the Life of the Digital Humanities

Digital Humanities Quarterly

Volume: 6

URL:

Additional Info:

The project this reports was winner of Outstanding Contribution Award from the Canadian Society for Digital Humanities

Ubiquitous Text Analysis

Poetess Archive Journal 2.1

Volume: 2

Book Chapters

Research access to in-copyright texts in the humanities

Information organization in digital humanities: Global perspectives

Editors: K. Golub, Y. Liu.

Volume: 1

Edition: 1st

URL:

Proceedings Paper

“The Library is Open!”: Open Data and an Open API for the HathiTrust Digital Library

Computational Humanities Research 2023, ceur-ws.org, Paris, France

Proceedings of the Computational Humanities Research Conference 2023

Editors: Artjoms Šeļa, Fotis Jannidis, Iza Romanowska.

Volume: 3558

URL:

Enriching Library Holdings for English Language Learners: Applying Traditional Readability Formulas and Modern Cohesion Methods to Film

Proceedings of the Association for Information Science and Technology

Volume: 60

Characterizing same work relationships in large-scale digital libraries

Information in Contemporary Society: iConference 2019

DOI:

Additional Info:

Winner, Best Short Research Paper

Access to billions of pages for large-scale text analysis

iConference 2017 Proceedings

Volume: 2

DOI:

Matching and grokking: Approaches to personalized crowdsourcing

Proceedings of the twenty-fourth international joint conference on artificial intelligence

Editors: Q. Yang, M. Wooldridge.

URL:

Additional Info:

Invited and peer-reviewed awards-track paper at a top artificial intelligent conference

Design facets of crowdsourcing

iConference 2015 Proceedings

A Crowd of Your Own: Crowdsourcing for On-Demand Personalization

Proceedings of the AAAI Conference on Human Computation and Crowdsourcing

Volume: 2

URL:

Additional Info:

Winner, Notable Paper Award. Featured in New Scientist

When the elevator pitch meets the subject heading: How mixtures of other documents can describe what a document is about

Proceedings of the American Society for Information Science and Technology

Volume: 51

DOI:

Additional Info:

Featured in Slate

Evaluating rater quality and rating difficulty in online annotation activities

Proceedings of the American Society for Information Science and Technology

Volume: 49

DOI:

Improving Retrieval of Short Texts Through Document Expansion

Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval

DOI:

Building Topic Models in a Federated Digital Library Through Selective Document Exclusion

Proceedings of the American Society for Information Science and Technology

Volume: 48

DOI:

Additional Info:

Best Paper

Semi-automated Collection Evaluation for Large-scale Aggregations

Proceedings of the American Society for Information Science and Technology

Reports

Similarities and Duplicates in Digital Libraries

Project Technical Report

DOI:

SIG-CM 2019: Workshop on Conceptual Modeling Report

DOI:

Exploring the Billions and Billions of Words in the HathiTrust Corpus with Bookworm

HathiTrust+Bookworm Project Technical Report

Extended Abstracts

Long paper abstracts at certain Humanities and Social Science venues, and poster extended-abstracts in Computer and Information Science proceedings.

Reducing Redundancy Bias in Digital Library Collections

Digital Humanities 2022

Challenges and Solutions of Identifying Similarities and Duplication in Digital Libraries

2020 ACM/IEEE Joint Conference on Digital Libraries

Examining patterns of text reuse in digitized text collections

2019 ACM/IEEE Joint Conference on Digital Libraries

DOI:

Worksets expand the scholarly utility of digital libraries

18th ACM/IEEE Joint Conference on Digital Libraries

DOI:

Modeling creativity: Tracking long-term lexical change

Digital Humanities 2017

URL:

Information-seeking in large-scale digital libraries: Strategies for scholarly workset creation

2017 ACM/IEEE Joint Conference on Digital Libraries

DOI:

Accomplishing low-attention microtasks

Productivity Decomposed: Getting Big Things Done with Little Microtasks

URL:

Improving consistency of crowdsourced multimedia similarity for evaluation

15th ACM/IEEE-CS Joint Conference on Digital Libraries

DOI:

Low Effort Crowdsourcing: Leveraging Peripheral Attention for Crowd Work

Second AAAI Conference on Human Computation & Crowdsourcing

Large-scale text analysis through the HathiTrust Research Center

Digital Humanities 2014

Using collections and worksets in large-scale corpora: Preliminary findings from the Workset Creation for Scholarly Analysis project

IConference 2014

URL:

Incidental Crowdsourcing: Crowdsourcing in the Periphery

Digital Humanities 2013

URL:

A Palette Mixing Model of Information Seeking for Complex Queries

Proceedings of the SIGIR 2011 Workshop on 'entertain Me': Supporting Complex Search Tasks

URL:

Structured Surfaces for JiTR

Digital Humanities 2011

When to ask for help: Evaluating projects for crowdsourcing

Digital Humanities 2011

A Day in the Life of Digital Humanities

Digital Humanities 2010

Mashing Texts: Supporting collections level text analysis

Digital Humanities 2009

Ubiquitous Text Analysis

Digital Humanities 2009

Presentations (refereed)

Conference presentations refereed from peer-reviewed short abstracts, as at Education conferences and some Social Science and Humanities conferences.

Extending Automated Scoring to the Figural Form of the Torrance Tests of Creative Thinking

AERA Annual Meeting

Beyond Human Raters: Using AI and NLP in PISA’s Creative Thinking Tests

NCME Annual Meeting

A Text-Mining Approach to Measuring Creativity

Annual Meeting, National Council for Measurement in Education

Text Mining in Educational Research: Demonstrating Language Models for Measuring Creativity

Methodology special interest group semi-annual meeting, European Association for Research on Learning and Instruction

A computational psychometric approach to measuring creative thinking

National Council on Measurement in Education

Using text-mining models to quantify creative thinking

National Council for Measurement in Education

Relational reasoning underlies creativity: Evidence from a text-mining investigation

Annual Meeting of the American Educational Research Association

A Modest Payment for a Modern Proposal

Annual Meeting of the Society for Digital Humanities

Pico Safari: Active Gaming in Integrated Environments

Annual Meeting of the Society for Digital Humanities

Additional Info:

Best Student Paper

Motivations of Users in Large-Scale Crowd-Powered Online Initiatives

Chicago Colloquium on Digital Humanities and Computer Science

What do we say about ourselves? An analysis of the Day of DH 2009 data

Annual Meeting of the Society for Digital Humanities

Shortcuts and Dead Ends: Control Issues With Online User-Generated Content

Annual Meeting of the Canadian Association for Information Science (CAIS)

Exploring New Possibilities in Rapid Research Document Management

Annual Meeting of the Society for Digital Humanities

Invasion of the And-Ors: Applying Active Learning Principles in the Development of a Flash-based Boolean Search Tutorial and Game

Annual Meeting of the Canadian Association for Information Science (CAIS)

Introducing a Day in the Life of the Digital Humanities

Annual Meeting of the Society for Digital Humanities (SDH/SEMI)

Presentations (invited)

Talking with Machines: The Do's and Don'ts of Effective Chatbot Use

Executive Energy Leadership Academy Alumni Event, National Renewable Energy Lab, Golden, CO

Applications of artificial intelligence to verbal and figural tests of creativity

International Society for the Science of Creativity and Innovation, presented remotely via video link

Let’s Go Beyond the Brick: Innovative Assessments of Creativity

Identifying Related and Same-Work Relationships in Large Digital Libraries

ASIS&T Webinars

Additional Info:

Location: Online
Sponsors: ASIS&T SIG-DM

Strategies for the utilisation of copyright-protected texts for third-party research

Non-consumptive Access to In-copyright Digital Works - the US Perspective

Additional Info:

Location: Trier, Germany
Sponsors: Institute for Digital Law Trier and the Trier Center for Digital Humanities, University of Trier

Tooling the future of massive-scale text research data

DataLab, University of Washington Information School

Additional Info:

Location: Seattle, Washington

HathiTrust Research Center

Nanjing University

Additional Info:

Location: Nanjing, China

Data Capsule in 7 minutes

Access Workshop 2017,Text Mining the Novel

Additional Info:

Location: Montreal

Presentations (other)

Characterizing Duplication and Similarity in Massive Digital Libraries

First Digital Front Range Symposium

Additional Info:

Location: University of Denver

Workshops (refereed)

Fairness and Accountability in Conceptual Models

Association for Information Science and Technology

References: Cancelled.

Workshop on conceptual models in digital libraries, archives, and museums

2020 ACM/IEEE Joint Conference on Digital Libraries

Workshop on conceptual models in digital libraries, archives, and museums

2019 ACM/IEEE Joint Conference on Digital Libraries

URL:

Teaching Workshops

Data Scholarship for the Humanities with the HathiTrust Digital Library

Additional Info:

Participants: 30 participants
Sponsors: Oxford e-Research Centre

HathiTrust Digital Library

Additional Info:

Participants: 40 participants
Sponsors: Beijing Institute of Technology

Panels

AI Today and Tomorrow in Gifted, Creativity, and Talent Research

AERA Annual Meeting

Classic and Novel Approaches in Creativity Assessment and Programming: Implications for Gifted Education

Annual Meeting, AERA, Chicago, IL

ChatGPT: The Future of Higher Ed and Libraries, Brought to You by AI

CNI Spring 2023 Membership Meeting, Coalition for Networked Information, Denver, CO, USA

Resilience in iSchools cohort education during a turbulent time: Improved normalcy through systemic inclusivity and other lessons learned to optimize virtual and physical spaces

iConference 2023, iSchools Consortium, Barcelona, Spain

Assessing the Research Data Management Landscape Through Practice and Education

Federation of Earth Science Information Partners, Online

Graduate Education in the Digital Humanities

Annual Meeting of the Society for Digital Humanities, Carleton University, Ottawa, Canada

Posters

Conference posters associated with a peer-reviewed short abstract.

Advancing Technologies for Traditional Library Services: Utilizing Content Based Recommendation to Assist in Readers' Advisory

World Library and Information Congress (WLIC), International Federation of Library Associations and Institutions, Dublin, Ireland

Conceptual Modeling of Similarities and Duplication in Large-Scale Digital Libraries

Digital Humanities 2020, Alliance for Digital Humanities Organization, Ottawa

MOODs: Identifying Outlier Passages and Texts in the Legislative Process

6th International Conference on Computational Social Science, Cambridge, MA/Online

Including Phrases in Bags of Words

Uncamp 2018, HathiTrust Research Center, Berkeley

Asking the Right Way: Design Interventions for Microtasks

The sixth AAAI Conference on Human Computation and Crowdsourcing, Association for Advancement of Artificial Intelligence, Quebec City, Canada

Personalized Human Computation

Conference on Human Computation and Crowdsourcing (HCOMP 2013), AAAI, Palm Springs, CA

Text Analysis for me Too: An embeddable text analysis widget

Annual Meeting of the Society for Digital Humanities, SDH/SEMI

TADA Research Evaluation Exchange: Winning 2008 Submissions

Digital Humanities 2009, University of Maryland, USA

Teaching

Courses

LIS 4015 - User and Access Services

3 credit hours

Format: in-person

LIS 4015 - User and Access Services

2 credit hours

Format: online

LIS 4220 - Data Curation

3 credit hours

LIS 4235 - Scripting for Large Databases

4 credit hours

Materials: Link

LIS 4700DH - Digital Humanities

3 credit hours

590TXT, LIS4991TXT - Text Mining

3 credit hours

Format: Independent study at DU

Materials: Link

Directed Student Learning, Dissertation Committee Member

  • Matthew Durward . "Audience Specific Text Recommendation for English Language Learners" University of Canterbury. (2021-)
  • Jill Talley. "Paving pathways for success: The role of transition models and disability services in postsecondary education for students with disabilities" (2021)
  • Paige Alfonso. "Exploring the relations between identity and digital media activity processes: A mixed-methods psychological investigation into connective action" (2019-2020)

Capstone, Committee Chair

  • Michele Newman. "A Taxonomy of Game Mechanics" (2022)
  • Sarah Beals. "Crowdsourced Data and Visualizing Theme Park Wait Times" (2020)
  • Matthew Durward. "Constructing a Platform to Assist English Language Learners with Film Selection" (2020)
  • Kevin Stevens. "Historiography as a Signature Pedagogical Approach to Information Literacy Instruction" (2020)
  • Rita Zhang. "A Framework of Active Exploratory Book Recommendation" (2020)
  • Lee Howick. "Faculty Opinions and Use of Open Access Publishing at the University of Denver" (2020)
  • Nicolas Pares. The Effects of Research Data Management Services: Associating the Data Curation Lifecycle with Open Research Output" (2020)
  • Vince Garin. "How Digital Humanities Challenges Publishing and Academic Labor Paradigms" (2019)
  • Micah Saxton. "Best Practices for Topic Modeling," Library & Information Science. (2018)
  • Sheena Young. "Library Thunderdome: Fake News vs. Information Literacy" (2018)

Capstone, Committee Member

  • Summer Shetenhelm. "Copyright Statements in Plains to Peaks Collective Digital Collections" (2019)
  • Fatih Gunaydin. "Assessment of Current Research Data Services in Terms of Technical Services" (2019)

Thesis, Committee Chair

  • Nicolas Pares. "Human-Centered Search Process: A Grounded Theory Study on the Modern Search Experience" (2022)

Non-Credit Instruction Taught

Guest Lecture

  • Topic: "Introduction to Docker for Data Curation". Data Curation, University of Illinois at Urbana-Champaign. Allen Renear. (Recorded for online course, 2018).
  • Topic: "Usability Design". Digital Libraries, University of Illinois at Urbana-Champaign, J. Stephen Downie.

Guest Lecture, DU

  • Topic: "Digital Humanities", Organization of Information, University of Denver. Krystyna Matusiak (2017-18, recorded for online course 2019-)
  • Topic: "Scholarly and Professional Communication", Foundations of Library and Information Science, University of Denver. Mary Stansbury (2017)

Faculty Development Activities Attended

OTL Faculty Workshop or Training, "Teaching@DU Short Course," Office of Teaching and Learning, Denver, CO, NF. (August 1, 2017 - December 31, 2017).

Professional Service

University Service

Committee Member, Open Access Fund. (June 2021 - Present).

Professional Memberships

Association for the Advancement of Artificial Intelligence. (November 2013 - Present).

American Society for Information Science and Technology. (2011 - Present).

Association for Computing Machinery. (2011 - Present).

Alliance of Digital Humanities Organizations. (2010 - Present).

Canadian Society for Digital Humanities. (2009 - 2013).

Canadian Association for Information Science. (2009 - 2011).

Committee Chair

Committee Chair, iSchools Conference Student Symposium Chair, Barcelona. (July 1, 2022 - March 2023).

Committee Member

Committee Member, Program Committee of Digital Humanities Workshop (DHW) 2021. (2021).

Committee Member, JCDL 2019 Outreach Committee. (May 2018 - June 2019).

Associate Chair, iConference 2019 Program Committee, College Park, Maryland. (June 2018 - February 2019).

Committee Member, HCOMP Program Committee. (January 2018 - October 2018).

Committee Member, JCDL Program Committee. (January 2018 - July 2018).

Guest Editor

Guest Co-Editor, Special Issue on "Conceptual Models of the Sociotechnical", Journal of Association for Information Science and Technology (JASIST) (2021-2022).

Editorial Review Board Member

Editorial Review Board Member, Przegląd Biblioteczny, Warsaw. (August 2021 - Present).

Editorial Review Board Member, Journal of Open Humanities Data. (July 2015 - Present).

Officer

President, Special Interest Group on Digital Libraries. (October 2020 - October 2021).

Conference Organizer

Organizer, First Digital Front Range Symposium, Digital Scholarship in the Front Range, Denver, Colorado. (May 5, 2018).

Reviewer, Grant Panelist

Panelist, National Endowment for the Humanities, Washington, D.C. (August 2018).

Reviewer, Conference Paper

Since 2017.

Association for Information Science and Technology. (2010 - 2019).

Digital Humanities. (2009 - 2021).

WebConf. (November 2018).

iConference. (September 2017 - October 2017).

Human-Centered Data Science and Social Computing. (October 2, 2017).

Reviewer, Journal Article

Since 2017.

  • Creativity Research Journal (2023-)
  • Behavioral Research Methods (2022-)
  • Journal of the Association for Information Science & Technology (2022).
  • Advances in Cognitive Psychology (2022).
  • Digital Humanities Quarterly. (April 2022).
  • International Journal for Digital Libraries. (March 2022 -- August 2022).
  • Data in Brief. (January 2022 - March 2022).
  • Journal of Open Humanities Data. (January 2022).
  • Symmetry. (January 2022).
  • Electronics (June 2022).
  • Mathematics. (December 2021).
  • Applied Sciences. (August 2021 - November 2021).
  • Entropy. (October 2021).
  • Telematics and Informatics. (June 2019).
  • Transactions on Information Systems. (October 2017).

Consulting

Higher Education, University of North Texas, Denton, TX. (May 2022 - July 2022).

Working with Dr. Selcuk Acar on developing a pilot of methods for scoring originality and elaboration in figural (i.e., drawn) tests of creativity.

Higher Education, Université du Québec à Montréal, Montreal, Canada. (August 2021).

Working with Dr. Pier-Luc de Chantel to develop an API to Open Creativity Scoring, making that system's scoring methods available to use in other tools and websites, such as Qualtrics.