| Question # | Question Text | Analysis | Data Sources | How user might query |
|---|
| 1 | Have my course grades declined over the years? | ANOVA of course grades, individual exam grades | Faculty uploaded spreadsheet with records of student grades for each year | Log-in as myself, check-off my courses that I am interested in analyzing |
| 2 | Why are my course grades declining? Are more students taking AP (bio) and placing out of this course? | Relationship among ACT/SAT scores, AP bio scores, course grades | Registrar demographic data: ACT/SAT scores, AP Bio scores; course grades; Faculty uploaded spreadsheet with records of student grades for each year | Log-in as myself, check-off my courses that I am interested in analyzing |
| 3 | Has the use of clickers improved student learning in my course? | Correlation (regression) between clicker use, clicker score(s) and exam scores | Registrar demographic data; Faculty uploaded exams, tagged by topic; Faculty uploaded clicker questions, tagged by topic; Faculty uploaded spreadsheet with: Annual records of student grades, Student grades on individual assessment items, Student grades or responses on individual clicker items | Log-in as myself, check-off my courses that I am interested in analyzing. Within courses, I will mark assessment items of interest (clicker questions on topic a, assessment questions on topic a) |
The following use cases and database requirements come from weekly meetings of the Database Development Team.
| Meeting Date | Use Case Description |
|---|
June 19, 2007
KBS | I would like to consolidate and archive my data and metadata. |
April 13, 2007
September 18, 2007 | I would like to tag these assessments with my own Blooms data |
April 13, 2007
September 18, 2007 | I would like to code these assessments according to my own taxonomy |
May 18, 2007
September 18, 2007 | I would like to add concept categories to this assessment |
| April 13, 2007 | I would like to know the answers and point values for this assessment item (or items) |
April 13, 2007
June 19, 2007 | I would like to know the grading rubric for this assessment item (or items) |
May 18, 2007
June 19, 2007 | I would like to find assessment items that address a particular misconception |
| June 19, 2007 | What is the learning objective associated with this assessment item? |
| May 18, 2007 | I would like to upload my non-MC exams and student responses.
Student responses are a scanned image -or- electronic. |
April 13, 2007
May 4, 2007 | Identifying information of students, faculty and universities must be removed |
| April 13, 2007 | I would like to import data directly from my course management software (e.g. Angel, Blackboard, WebCT, etc) |
| May 4, 2007 | I would like to import student response data directly from my excel workbook |
April 13, 2007
September 18, 2007 | There must be a minimum level of data/metadata included for a submission to be accepted into the database |
| May 18, 2007 | What areas or concepts in biology/chemistry/physics are students having difficulty with? |
| May 18, 2007 | How did students do on all questions of a given topic?
-within a class
-within a course
-within an institution
-across the database |
| May 18, 2007 | How did students do on a question that required a certain level of thinking? |
| May 18, 2007 | I would like to find those questions on topic X where students did particularly well or poorly (depending on my own criteria of good and bad) |
| May 18, 2007 | Compare performance of two (or more) groups of students on an item or set of items
-over time (this year versus last year)
-gender or ethnic background
-majors or non-majors |
| May 18, 2007 | Find questions based on format (clicker, mc, short answer, etc) |
| May 18, 2007 | Find questions that include a particular format element (i.e. graph, particular model or image, etc). |
| June 12, 2007 | I would like to query across data levels (i.e. query for particular assessment items at an institution |
| June 19, 2007 | I would like demographic data returned |
| June 19, 2007 | I would like to see what concepts are considered difficult across disciplines |
| July 3, 2007 | I have additional data/metadata I would like to upload - where can I do that? |
| July 3, 2007 | Assessment items will be tagged according to an ontology, not by categories. |
| July 31, 2007 | I would like to upload my scientific reasoning instrument.
Questions do not fit a discipline-based ontology
This instrument is administered at multiple institutions. |
| August 30, 2007 | Each student, class, course, institution, faculty member must have a unique identifier |
| August 30, 2007 | Faculty members must be able to indicate pages on an exam that are blank, devoted to a figure, picture or formula |
| August 30, 2007 | Who can access the database? (registered users, anyone? What level of data is returned?) |
| August 30, 2007 | The database must handle multiple instructors for a class -and- visiting instructors (i.e. faculty who are not officially on staff at an institution) |
| September 11, 2007 | Database must deal with multiple sections of a single course. |
| October 9, 2007 | Data is returned completely anonymized/partially anonymized/not anonymized. |
| October 9, 2007 | I would like to track a cohort of students over time. |
| October 9, 2007 | I would like to upload student response data, but some answers contain identifying information. |
| November 6, 2007 | Demographic data will be stored with individual assessments. Students will not have global identifiers. |
| November 6, 2007 | Query returns must be constrained so as not to return a limited dataset that could result in the identification of a student. |
| November 6, 2007 | I made a mistake in the classification of an assessment item. I'd like to revise it. |
| November 6, 2007 | I would like to find assessment items tagged as 'good' - search quality tags by a professional society. |
| November 6, 2007 | I have no idea how to classify my assessment items. |
| November 27, 2007 | User will be able to search nbii thesaurus and with a mouse click, add that concept tag to their assessment item(s) |
| November 27, 2007 | Database will use the NBII Biocomplexity thesaurus as the Biology ontology |
| November 27, 2007 | Ontology for a discipline must not be rigid - additions must be possible. |
| December 4, 2007 | I would like data returned in a simple, rectangular format. |
| December 4, 2007 | I would like data returned in some other formant ??? |
| December 4, 2007 | I would like to query all R1 institutions, based on the Carnegie classification system. |
| December 4, 2007 | I would like a simple analysis of change over time returned with the data. |
| December 4, 2007 | I would like my search history saved. |
| December 4, 2007 | I would like the details of my query returned with the results. |
| January 10, 2008 | Instead of answering a litany of questions, I will upload my course syllabus. |
| January 10, 2008 | I would like my syllabus to be anonymized when uploaded. |
| KBS | I would like to filter out low-level Bloom questions, multiple choice questions, essay questions, etc. |
| KBS | I would like to find other faculty who are similar to me in teaching approach/assessment items/Bloom's level/etc |
| KBS | I would like to upload a published assessment (e.g. Concept Inventory) |
| KBS | Database must store unconventional assessments (models, pictures, diagrams, etc). |
| KBS | All data or a standard subset of data and metadata should be reported for each assessment item. |
| Purdue | I am an administrator and would like to use the database. |
| Purdue | Administrator: would like to compare new faculty with tenured faculty based on:
-student scores
-average Bloom level per exam
-average number of students taught a semester
-course level taught |
