| Removed lines 3-26 |
| - !Research Questions |
| - ||Question #||Question Text||Analysis||Data Sources||How user might query |
| - |1|[Have my course grades declined over the years?|UseCase1]|ANOVA of course grades, individual exam grades|Faculty uploaded spreadsheet with records of student grades for each year|Log-in as myself, check-off my courses that I am interested in analyzing |
| - |2|[Why are my course grades declining? Are more students taking AP (bio) and placing out of this course?|UseCase2]|Relationship among ACT/SAT scores, AP bio scores, course grades|Registrar demographic data: ACT/SAT scores, AP Bio scores; course grades; Faculty uploaded spreadsheet with records of student grades for each year|Log-in as myself, check-off my courses that I am interested in analyzing |
| - |3|Has the use of clickers improved student learning in my course?|Correlation (regression) between clicker use, clicker score(s) and exam scores|Registrar demographic data; Faculty uploaded exams, tagged by topic; Faculty uploaded clicker questions, tagged by topic; Faculty uploaded spreadsheet with: Annual records of student grades, Student grades on individual assessment items, Student grades or responses on individual clicker items|Log-in as myself, check-off my courses that I am interested in analyzing. Within courses, I will mark assessment items of interest (clicker questions on topic a, assessment questions on topic a) |
| - |4|What is the impact of multiple attempts at online homework on exam scores in my course?|Correlation between number of tries and exam scores\\Regression/partial regression using GPA or other control variables|Registrar demographic data\\Registrar GPA data\\Faculty uploaded exams, tagged by topic\\Faculty uploaded spreadsheet with:\\Student exam scores\\Student grades on individual items|Log-in as myself, check-off my courses that I am interested in analyzing\\With-in each course, I will mark assessment items of interest (homework questions on topic a, assessment questions on topic a) |
| - |5|How are my students performing on a particular topic?\\Are they doing better on evolution this year (versus last year)?|Pre-test/post-test analysis\\ANOVA of grades on specific assessment items|Registrar demographic data\\Faculty uploaded assessments and rubric for each year, tagged by topic\\Faculty uploaded spreadsheet with:\\Student exam scores for each year, Student grades on individual assessment items for each year|Log-in as myself, check-off my courses that I am interested in analyzing\\Within each course, I will mark assessment items of interest |
| - |6|How are my students performing on questions that are of a higher Bloom’s level?| Descriptive statistics on questions based on classifications – correlation\\Cluster analysis (PCA) of outcomes based on classification, topic, and the interaction of those two|Registrar demographic data\\Faculty uploaded assessments and rubric, tagged with Bloom’s data\\Faculty uploaded spreadsheet with: Student exam scores for each year; Student grades on individual assessment items for each year| Log-in as myself, check-off my courses that I am interested in analyzing\\Within each course, I will mark assessment items of interest to me, based on Bloom’s levels |
| - |7|Does a particular teaching innovation (i.e. Avida-ED) impact student learning of a particular topic (i.e. Evolution) in my course?|Correlation between innovation and exam score (or score on particular assessment items on exam) over time|Registrar demographic data\\Faculty uploaded assessments and rubric for each year, tagged by topic\\Faculty uploaded spreadsheet with: Student exam scores for each year; Student grades on individual assessment items for each year| Log-in as myself, check-off my courses that I am interested in analyzing\\Within each course, I will mark assessment items of interest to me, based on keywords or tags\Demographic data\\Rubric for exam or assessment items\\Student response data for exam or assessment items (at T0 and T1)\\Student scores for exam or assessment items (at T0 and T1) |
| - |8|Is there a difference in course grades between sections? (e.g. Different sections could be taught by different TAs, at different times, etc)|Correlation between section and individual assessment (or assessment items)/course grades (or between TA and section grades)|Registrar demographic data\\Faculty uploaded assessments and rubric for each year, tagged by topic\\Faculty uploaded spreadsheet with: Student exam scores; Student grades on individual assessment items|Log-in as myself, check-off my courses that I am interested in analyzing |
| - |9|How do my students compare with students from another school on topic x?|Correlation between institution and grade on topic x.|Registrar demographic data\\Faculty uploaded assessments and rubrics, tagged by topic\\Faculty uploaded spreadsheet with: Student exam scores; Student grades on individual assessment items\\Metadata linking course to a particular type/sized institution|Log-in as myself, check off my courses that I am interested in analyzing\\Select my assessment items from topic x\\Search database for additional school to analyze that are similar in Carnegie categorization, similar in course size, all schools\\Search these assessments for items on topic x |
| - |10|For accreditation, I would like to show that our students have learned y.|Comparison of student performance at T0 and T1.|Registrar demographic data\\Faculty uploaded assessments and rubric, tagged with topic and Bloom’s data\\Faculty uploaded spreadsheet with: Student exam scores for each year; Student grades on individual assessment items for each year|Log-in as myself, check off my courses that I am interested in analyzing\\Select my assessment items from topic x |
| - |11|What levels of understanding are typically targeted by assessments in introductory (or upper) level biology courses at my institution/at institutions similar to mine/at all institutions?|Proportion of lower-level Bloom’s items to higher-level Bloom’s items\\Histogram of data| Faculty uploaded assessments, tagged by Bloom’s data\\Metadata form tagging course level, institution information (size, type)|Search for all assessment items in courses at the 100-level |
| - |12|What misconceptions do students generally hold about topic d?|Qualitative analysis of student responses and scores to parse out misconceptions| Registrar demographic data\\Faculty uploaded assessments tagged with topic and Bloom’s data| Search for all assessment items on topic d |
| - |13|What is the impact of course size on the assessments used?|Correlation between course size and Bloom’s taxonomy level| Registrar demographic data\\Faculty uploaded assessments tagged with topic and Bloom’s data\\Metadata form indicating course size|Search for courses of a particular size |
| - |14|Are low performers in my course also struggling in other courses?|Correlation between course grade or individual assessment grade and university GPA, major GPA| Registrar demographic data\\Registrar GPA data\\Faculty uploaded spreadsheet with student course grades| Log-in as myself, check off my courses that I am interested in analyzing |
| - |15|How reliable is my exam question taxonomy?| |
| - |May 18, 2007|What areas or concepts in biology/chemistry/physics are students having difficulty with? |
| - |May 18, 2007|How did students do on all questions of a given topic?\\-within a class\\-within a course\\-within an institution\\-across the database |
| - |May 18, 2007|How did students do on a question that required a certain level of thinking? |
| - |May 18, 2007|I would like to find those questions on topic X where students did particularly well or poorly (depending on my own criteria of good and bad) |
| - |May 18, 2007|Compare performance of two (or more) groups of students on an item or set of items\\-over time (this year versus last year)\\-gender or ethnic background\\-majors or non-majors |
| - |
| - ---- |
| At line 27 added 1 line. |
| + |
| Line 59 was replaced by line 36 |
| - |31|June 19, 2007|Query: What conceptsare considered difficult across disciplines. Will the database compute dynamic metadata such as the difficulty of an item? Such data would change based on the data from the students. Difficulty and other performance metrics should be dynamic, rather than asking faculty to upload them.|pending - will we provide simple analysis?|x| |x| |x| |Nice |
| + |31|June 19, 2007|Query: What concepts are considered difficult across disciplines. Will the database compute dynamic metadata such as the difficulty of an item? Such data would change based on the data from the students. Difficulty and other performance metrics should be dynamic, rather than asking faculty to upload them.|pending - will we provide simple analysis?|x| |x| |x| |Nice |
| Line 66 was replaced by lines 43-65 |
| - |
| + ---- |
| + !Research Questions |
| + ||Question #||Question Text||Analysis||Data Sources||How user might query |
| + |1|[Have my course grades declined over the years?|UseCase1]|ANOVA of course grades, individual exam grades|Faculty uploaded spreadsheet with records of student grades for each year|Log-in as myself, check-off my courses that I am interested in analyzing |
| + |2|[Why are my course grades declining? Are more students taking AP (bio) and placing out of this course?|UseCase2]|Relationship among ACT/SAT scores, AP bio scores, course grades|Registrar demographic data: ACT/SAT scores, AP Bio scores; course grades; Faculty uploaded spreadsheet with records of student grades for each year|Log-in as myself, check-off my courses that I am interested in analyzing |
| + |3|Has the use of clickers improved student learning in my course?|Correlation (regression) between clicker use, clicker score(s) and exam scores|Registrar demographic data; Faculty uploaded exams, tagged by topic; Faculty uploaded clicker questions, tagged by topic; Faculty uploaded spreadsheet with: Annual records of student grades, Student grades on individual assessment items, Student grades or responses on individual clicker items|Log-in as myself, check-off my courses that I am interested in analyzing. Within courses, I will mark assessment items of interest (clicker questions on topic a, assessment questions on topic a) |
| + |4|What is the impact of multiple attempts at online homework on exam scores in my course?|Correlation between number of tries and exam scores\\Regression/partial regression using GPA or other control variables|Registrar demographic data\\Registrar GPA data\\Faculty uploaded exams, tagged by topic\\Faculty uploaded spreadsheet with:\\Student exam scores\\Student grades on individual items|Log-in as myself, check-off my courses that I am interested in analyzing\\With-in each course, I will mark assessment items of interest (homework questions on topic a, assessment questions on topic a) |
| + |5|How are my students performing on a particular topic?\\Are they doing better on evolution this year (versus last year)?|Pre-test/post-test analysis\\ANOVA of grades on specific assessment items|Registrar demographic data\\Faculty uploaded assessments and rubric for each year, tagged by topic\\Faculty uploaded spreadsheet with:\\Student exam scores for each year, Student grades on individual assessment items for each year|Log-in as myself, check-off my courses that I am interested in analyzing\\Within each course, I will mark assessment items of interest |
| + |6|How are my students performing on questions that are of a higher Bloom’s level?| Descriptive statistics on questions based on classifications – correlation\\Cluster analysis (PCA) of outcomes based on classification, topic, and the interaction of those two|Registrar demographic data\\Faculty uploaded assessments and rubric, tagged with Bloom’s data\\Faculty uploaded spreadsheet with: Student exam scores for each year; Student grades on individual assessment items for each year| Log-in as myself, check-off my courses that I am interested in analyzing\\Within each course, I will mark assessment items of interest to me, based on Bloom’s levels |
| + |7|Does a particular teaching innovation (i.e. Avida-ED) impact student learning of a particular topic (i.e. Evolution) in my course?|Correlation between innovation and exam score (or score on particular assessment items on exam) over time|Registrar demographic data\\Faculty uploaded assessments and rubric for each year, tagged by topic\\Faculty uploaded spreadsheet with: Student exam scores for each year; Student grades on individual assessment items for each year| Log-in as myself, check-off my courses that I am interested in analyzing\\Within each course, I will mark assessment items of interest to me, based on keywords or tags\Demographic data\\Rubric for exam or assessment items\\Student response data for exam or assessment items (at T0 and T1)\\Student scores for exam or assessment items (at T0 and T1) |
| + |8|Is there a difference in course grades between sections? (e.g. Different sections could be taught by different TAs, at different times, etc)|Correlation between section and individual assessment (or assessment items)/course grades (or between TA and section grades)|Registrar demographic data\\Faculty uploaded assessments and rubric for each year, tagged by topic\\Faculty uploaded spreadsheet with: Student exam scores; Student grades on individual assessment items|Log-in as myself, check-off my courses that I am interested in analyzing |
| + |9|How do my students compare with students from another school on topic x?|Correlation between institution and grade on topic x.|Registrar demographic data\\Faculty uploaded assessments and rubrics, tagged by topic\\Faculty uploaded spreadsheet with: Student exam scores; Student grades on individual assessment items\\Metadata linking course to a particular type/sized institution|Log-in as myself, check off my courses that I am interested in analyzing\\Select my assessment items from topic x\\Search database for additional school to analyze that are similar in Carnegie categorization, similar in course size, all schools\\Search these assessments for items on topic x |
| + |10|For accreditation, I would like to show that our students have learned y.|Comparison of student performance at T0 and T1.|Registrar demographic data\\Faculty uploaded assessments and rubric, tagged with topic and Bloom’s data\\Faculty uploaded spreadsheet with: Student exam scores for each year; Student grades on individual assessment items for each year|Log-in as myself, check off my courses that I am interested in analyzing\\Select my assessment items from topic x |
| + |11|What levels of understanding are typically targeted by assessments in introductory (or upper) level biology courses at my institution/at institutions similar to mine/at all institutions?|Proportion of lower-level Bloom’s items to higher-level Bloom’s items\\Histogram of data| Faculty uploaded assessments, tagged by Bloom’s data\\Metadata form tagging course level, institution information (size, type)|Search for all assessment items in courses at the 100-level |
| + |12|What misconceptions do students generally hold about topic d?|Qualitative analysis of student responses and scores to parse out misconceptions| Registrar demographic data\\Faculty uploaded assessments tagged with topic and Bloom’s data| Search for all assessment items on topic d |
| + |13|What is the impact of course size on the assessments used?|Correlation between course size and Bloom’s taxonomy level| Registrar demographic data\\Faculty uploaded assessments tagged with topic and Bloom’s data\\Metadata form indicating course size|Search for courses of a particular size |
| + |14|Are low performers in my course also struggling in other courses?|Correlation between course grade or individual assessment grade and university GPA, major GPA| Registrar demographic data\\Registrar GPA data\\Faculty uploaded spreadsheet with student course grades| Log-in as myself, check off my courses that I am interested in analyzing |
| + |15|How reliable is my exam question taxonomy?| |
| + |May 18, 2007|What areas or concepts in biology/chemistry/physics are students having difficulty with? |
| + |May 18, 2007|How did students do on all questions of a given topic?\\-within a class\\-within a course\\-within an institution\\-across the database |
| + |May 18, 2007|How did students do on a question that required a certain level of thinking? |
| + |May 18, 2007|I would like to find those questions on topic X where students did particularly well or poorly (depending on my own criteria of good and bad) |
| + |May 18, 2007|Compare performance of two (or more) groups of students on an item or set of items\\-over time (this year versus last year)\\-gender or ethnic background\\-majors or non-majors |
