| Lines 29-82 were replaced by lines 29-64 |
| - |1|June 19, 2007\\KBS|I would like to consolidate and archive my data and metadata.|n/a| | |x|x| | |Must |
| - |2|April 13, 2007\\September 18, 2007\\June 19, 2007|Tagging assessment items: with Blooms data, concept categories, learning objective, some other taxonomy|yes - will support| | |x| | | |Must |
| - |3|3-Jul-07|Assessment items will be tagged according to an ontology|yes|x| |x| |x| |Must |
| - |4|27-Nov-07|Database will use the NBII Biocomplexity thesaurus as the Biology ontology.|yes|x| |x| | | |Must |
| - |5|27-Nov-07|Ontology for a discipline must not be rigid - additions must be possible.|yes?|x| |x| | |x|Must |
| - |6|February 4, 2008\\MU-L|Must be able to use different content metadata by discipline. While we can use the NBII thesaurus for biology, we need to have an API that can be used to define other metadata for other disciplines.|yes - will support| | |x| | |x|Must |
| - |7|February 4, 2007\\MU-L\\JLM|This assessment (or assessment item) has multiple rubrics)|yes - will support|x| |x| | | |Must |
| - |8|February 4, 2008\\MU-L\\JLM|This assessment has been scored in multiple ways (see above).|yes - will support| | |x| | | |Must |
| - |9|February 4, 2008\\MU-L\\JLM|This assessment has multiple and different raters on two (or more) different rubrics|yes - will support| | |x| | | |Must |
| - |10|13-Apr-07|I would like to see the raw response data for an assessment item(s)\\I would like to see the points assigned to a particular response|yes - will support| | |x| | | |Must |
| - |11|May 18, 2007\\June 19, 2007|I would like to find assessment items that address a particular misconception|n/a|x| |x| | | |Must |
| - |12|May 18, 2007\\October 9, 2007|I would like to upload my non-MC exams and student responses.\\Student responses are a scanned image -or- electronic.|yes - will support| | | | | | |Must |
| - |13|7-Feb-08|Faculty who upload data must agree that any identifying information in student responses has been removed|yes - will support -- how?| | | | | |x|Must |
| - |14|July 3, 2007\\November 6, 2007|I have additional data/metadata I would like to upload. -or- I made a mistake. Where can I do that?|n/a|x|x|x|x| | |Must |
| - |15|April 13, 2007\\May 4, 2007|Identifying information of students, faculty and universities must be removed|yes - will support| | | |x| | |Must |
| - |16|4-May-07|I would like to import student response data directly from my excel workbook|yes - will support| | | | | |x|Must |
| - |17|April 13, 2007\\September 18, 2007\\KBS|There must be a minimum level of data/metadata included for a submission to be accepted into the database|Must define minimum metadata standards| | | | | |x|Must |
| - |18|19-Jun-07|I would like demographic data returned.|yes - will support| | | |x| | |Must |
| - |19|31-Jul-07|I would like to upload my scientific reasoning instrument.\\Questions do not fit a discipline-based ontology\\This instrument is administered at multiple institutions.|yes|x|x|x| |x| |Must |
| - |20|30-Aug-07|Each student, class, course, institution, faculty member must have a unique identifier.|yes - will support| | | | |x|x|Must |
| - |21|30-Aug-07|Faculty members must be able to indicate pages on an exam that are blank, devoted to a figure, picture or formula|yes - completed?| | | | | |x|Must |
| - |22|11-Sep-07|Database must deal with multiple sections of a single course.|yes| | | |x|x| |Must |
| - |23|9-Oct-07|Data is returned completely anonymized/partially anonymized/not anonymized.|yes - completely anonymized| | | |x| |x|Must |
| - |24|9-Oct-07|I would like to track a cohort of students over time.|yes - will support| | | |x|x| |Must |
| - |25|6-Nov-07|Query returns must be constrained so as not to return a limited dataset that could result in the identification of a student.|yes - will support| | | |x|x| |Must |
| - |26|4-Dec-07|I would like data returned in a simple, rectangular format.|yes - will support| | | | | |x|Must |
| - |27|10-Jan-08|Instead of answering a litany of questions, I will upload my course syllabus.|pending| | | |x| |x|Must |
| - |28|KBS|I would like to upload a published assessment (e.g. Concept Inventory).|pending|x|x|x| | | |Must |
| - |29|KBS|Database must store unconventional assessments (models, pictures, diagrams, etc).|yes - will support|x| | | | |x|Must |
| - |30|4-Feb-08|How will assessment items be returned? Will users pick out one question at a time, or will they check off those questions they are interested in downloading? What format will questions be returned - xml, QTI, etc?|pending|x| | |x| | |Must |
| - |31|February 11, 2008\\MU-L\\JLM|Demographic data - as uploaded by a faculty member - needs to be tagged with the date the data were current|pending| | | | | |x|Must |
| - |32|April 13, 2007\\May 22, 2007|Rubric: I would like to upload or download a rubric for this assessment item(s)|yes - will support|x| |x| | | |Must |
| - |33|7-Feb-08|User uploading data takes responsibility for anonymizing their syllabus.|yes - will support| |x| | | | |Must |
| - |34|6-Nov-07|I have no idea how to classify my assessment items.|no| |x| | | | |Must - user documentation |
| - |36|18-May-07|Find questions based on format (clicker, mc, short answer, etc)|pending -- can this information be gleaned from the parsing software?|x| |x| |x| |Nice |
| - |37|18-May-07|Find questions that include a particular format element (i.e. graph, particular model or image, etc).|outside current scope| | | | | | |Nice |
| - |38|19-Jun-07|I would like to see what concepts are considered difficult across disciplines. Are we going to compute dynamic meta data such as the difficulty of an item that would change based on the data from the students? Difficulty and other performance metrics should be dynamic, rather than asking faculty to upload them.|pending - will we provide simple analysis?|x| |x| |x| |Nice |
| - |39|4-Dec-07|I would like a simple analysis of change over time returned with the data.|pending| | | |x|x| |Nice |
| - |40|10-Jan-08|I would like my syllabus to be anonymized when uploaded.|pending| | | |x| | |Nice |
| - |41|February 4, 2008\\MU-L\\JLM|Will the syllabus be query-able? Doesn't seem likely. Need to revisit minimum metadata standards.|pending| | | | | | |Nice |
| - |42|Purdue|I am an administrator and would like to use the database.|pending| | | | | | |Nice |
| - |43|Purdue|Administrator: would like to compare faculty based on:\\-student scores\\-average Bloom level per exam\\-average number of students taught a semester\\-course level taught|n/a| | |x|x|x| |Nice |
| - |44|27-Nov-07|User will be able to search nbii thesaurus and with a mouse click, add that concept tag to their assessment item(s).|yes?|x| |x| | | |Should |
| - |45|13-Apr-07|I would like to import data directly from my course management software (e.g. Angel, Blackboard, WebCT, etc)|outside current scope| | | | | |x|Should |
| - |46|30-Aug-07|Who can access the database? -- One solution: Two levels of data: Personal data (visible only to the individual who uploaded) and Public (visible to everyone)|pending|x|x| | | | |Should |
| - |47|30-Aug-07|The database must handle multiple instructors for a class -and- visiting instructors (i.e. faculty who are not officially on staff at an institution)|no| | | |x|x| |Should |
| - |48|6-Nov-07|I would like to find assessment items tagged as 'good' - search quality tags by a professional society.|yes - will support|x| |x| |x| |Should |
| - |49|4-Dec-07|I would like to query all R1 institutions, based on the Carnegie classification system.|pending| | | | |x| |Should |
| - |50|4-Dec-07|I would like my search history saved. Are we saving the query syntax or are we expecting to be able to reproduce the exact dataset? Can probably reproduce the dataset if the date that the data were uploaded is stored in the database and can be made part of the query.|pending| | | | |x|x|Should |
| - |51|4-Dec-07|I would like the details of my query returned with the results.|pending| | | | | |x|Should |
| - |52|KBS|I would like to filter out low-level Bloom questions, multiple choice questions, essay questions, etc.|pending| x| | | |x| |Should |
| - |53|KBS|I would like to find other faculty who are similar to me in teaching approach/assessment items/Bloom's level/etc.|pending| | |x|x|x| |Should |
| - |54|February 4, 2008\\MU-L\\JLM|Administrator would like to show their program has met specific accreditation standards.|n/a| | |x|x|x| |Should |
| - |55|12-Jun-07|I would like to query across data levels (i.e. query for particular assessment items at an institution over time and courses|yes - will support| | |x|x|x|| |
| + |1|April 13, 2007\\September 18, 2007\\June 19, 2007|Tagging assessment items: with Blooms data, concept categories, learning objective, some other taxonomy\\tag in multiple ways\\tag with multiple raters|yes - will support| | |x| | | |Must |
| + |2|July 3, 2007|Assessment items will be tagged according to a domain specific ontology\\Biology ontology: NBII Biocomplexity Thesaurus\\Ontologies must not be rigid -- must allow for additions.|yes - will support|x| |x| |x| |Must |
| + |3|February 4, 2007\\MU-L\\JLM|Rubrics: database must support multiple rubrics for an assessment item\\multiple scores for an assessment item\\multiple raters of an assessment item|yes - will support|x| |x| | | |Must |
| + |4|April 13, 2007|Data storage: Must store raw response data for assessment items\\point values for assessment items and assessments (i.e., entire exams)\\student grade on assessment item|yes - will support| | |x| | | |Must |
| + |5|December 4, 2007|Data return format: rectangular|yes - will support| | | | | |x|Must |
| + |6|February 4, 2008|Return of assessment items: Will users pick out one question at a time, or will they check off those questions they are interested in downloading? What format will questions be returned - xml, QTI, etc?|pending|x| | |x| | |Must |
| + |7|May 18, 2007\\October 9, 2007|Data upload: non-MC exams and student responses.\\Student responses are a scanned image -or- electronic.|yes - will support| | | | | | |Must |
| + |8|February 7, 2008|Security: Faculty who upload data must agree that any identifying information in student responses has been removed\\any identifying information in their syllabus that they do not wish to be published in the dBase has been removed.|yes - will support -- how?| | | | | |x|Must |
| + |9|April 13, 2007\\May 4, 2007|Security: Student ids will be hashed with a unique identifier\\Faculty and university ids could also be hashed.\\data is returned completely anonymous\\queries are constrained so as to not return a limited dataset that could identify a student.|yes - will support| | | ||x|x|Must |
| + |10|July 3, 2007\\November 6, 2007|I have additional data/metadata I would like to upload. Where can I do that?|yes - will support|x|x|x|x| | |Must |
| + |11|February 18, 2008\\MU-L\\JLM|Editing environment for data: how can faculty fix errors in data they have uploaded?|yes - will support||||||x|Must |
| + |12|April 13, 2007\\September 18, 2007\\KBS|Minimum metadata requirements: We must define the minimum level of data/metadata included for a submission to be accepted into the database\\Will uploading a syllabus (which is not searchable) meet metadata requirements?|Must define minimum metadata standards| | | | | |x|Must |
| + |13|May 4, 2007|I would like to import student response data directly from my excel workbook|yes - will support| | | | | |x|Must |
| + |14|June 19, 2007|Demographic data will be available for all queries.\\Faculty uploaded demographic data must be tagged with the date they were current.|yes - will support| | | |x| | |Must |
| + |15|August 30, 2007|The database must handle multiple instructors for a class.|no| | | |x|x| |Must |
| + |16|August 30, 2007|Faculty members must be able to indicate pages on an exam that are blank, devoted to a figure, picture or formula|yes - completed?| | | | | |x|Must |
| + |17|September 11, 2007|Database will treat sections of a course as separate courses.\\How can we alleviate work for a faculty member uploading the same assessment for 60 sections of a very large course?|yes| | | |x|x| |Must |
| + |18|October 9, 2007|Query: track students over time.|yes - will support| | | |x|x| |Must |
| + |19|May 18, 2007\\June 19, 2007|Metadata query: Query for common misconceptions|yes - will support|x| |x| | | |Must |
| + |20|July 31, 2007|Non-traditional assessments: Scientific reasoning instruments, concept inventories, published assessment\\Assessment items will not fit a discipline-based ontology|yes|x|x|x| |x| |Must |
| + |21|June 12, 2007|I would like to query across data levels (i.e. query for particular assessment items at an institution over time and courses|yes - will support| | |x|x|x||Must |
| + |22|August 30, 2007|Who can access the database? -- One solution: Two levels of data: Personal data (visible only to the individual who uploaded) and Public (visible to everyone)|pending|x|x| | | | |Should |
| + |23|November 6, 2007|Multiple taxonomies: What is the scope? Who can use a particular taxonomy -- public (i.e., Blooms) and private taxonomies (i.e., professional society tags on 'good' assessment items)?|yes - will support|x| |x| |x| |Should |
| + |24|December 4, 2007|Query: Search database on a paticular type of institution, based on the Carnegie classification system.|pending| | | | |x| |Should |
| + |25|December 4, 2007|Save search history syntax in order to repeat search at some point in the future.\\Alternatively, return search syntax with query results.|pending| | | | |x|x|Should |
| + |26|KBS|I would like to filter out low-level Bloom questions, multiple choice questions, essay questions, etc.|pending| x| | | |x| |Should |
| + |27|KBS|Fostering collaboration: will faculty information be available? Can faculty decide whether to make their contact information available?|pending| | |x|x|x| |Should |
| + |28|KBS|Unconventional assessment items: database will store the item\\Student response data (excel, word files, etc) will also be stored.|yes - will support|x| | | | |x|Nice |
| + |29|November 6, 2007|I have no idea how to classify my assessment items.|no| |x| | | | |Nice |
| + |30|May 18, 2007|Query: Find questions based on format (clicker, mc, short answer, etc)\\Find questions that include a particular format element (graph, image, model)|pending -- can this information be gleaned from the parsing software?|x| |x| |x| |Nice |
| + |31|June 19, 2007|Query: What conceptsare considered difficult across disciplines. Will the database compute dynamic metadata such as the difficulty of an item? Such data would change based on the data from the students. Difficulty and other performance metrics should be dynamic, rather than asking faculty to upload them.|pending - will we provide simple analysis?|x| |x| |x| |Nice |
| + |32|December 4, 2007|Data analysis: simple analysis of change over time returned with the data.|pending| | | |x|x| |Nice |
| + |33|January 10, 2008|Syllabus: Will it be query-able? Probably not.\\Will it be anonymized on upload? Probably not. Faculty will take responsibility for anonymization.|pending| | | |x| | |Nice |
| + |34|Purdue|Who can access data across courses/faculty members? Adiminstrator use to demonstrate compliance of accreditation standards.|pending| | | | | | |Nice |
| + |35|November 27, 2007|User will be able to search nbii thesaurus and with a mouse click, add that concept tag to their assessment item(s).|yes?|x| |x| | | |Nice |
| + |36|April 13, 2007|I would like to import data directly from my course management software (e.g. Angel, Blackboard, WebCT, etc)|outside current scope| | | | | |x|Nice |
| At line 84 added 2 lines. |
| + |
| + |
