| Title: | Tools and Methods for Working with MIDFIELD Data in 'R' |
|---|---|
| Description: | Provides tools in 'R' for working with undergraduate, longitudinal, student-level records modeled on the MIDFIELD database. Tools facilitate identifying academic program codes, excluding post-baccalaureate terms, excluding records for insufficient data, and assessing timely completion. The tools support the workflow of collecting programs, refining the population, constructing blocs of records for aggregation, and calculating quantitative metrics. 'midfieldr' interacts with practice data provided in the 'midfielddata' package or with any data modeled on the MIDFIELD database. The development of 'midfieldr' and 'midfielddata' was supported by the US National Science Foundation through grant numbers 1545667 and 2142087. |
| Authors: | Richard Layton [cre, aut, cph], Russell Long [aut, cph], Matthew Ohland [aut, cph], Marisa Orr [aut, cph], Susan Lord [aut, cph], US National Science Foundation [fnd] |
| Maintainer: | Richard Layton <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 1.0.3.9021 |
| Built: | 2026-07-14 01:09:56 UTC |
| Source: | https://github.com/midfieldr/midfieldr |
These functions were deprecated in midfieldr 1.0.4.
add_completion_status(dframe, midfield_degree = degree) add_data_sufficiency(dframe, midfield_term = term) filter_cip(keep_text = NULL, drop_text = NULL, cip = NULL, select = NULL) select_required(midfield_x, select_add = NULL) add_timely_term( dframe, midfield_term = term, ..., sched_span = NULL, span = NULL )add_completion_status(dframe, midfield_degree = degree) add_data_sufficiency(dframe, midfield_term = term) filter_cip(keep_text = NULL, drop_text = NULL, cip = NULL, select = NULL) select_required(midfield_x, select_add = NULL) add_timely_term( dframe, midfield_term = term, ..., sched_span = NULL, span = NULL )
dframe |
Data frame or data frame extension (e.g., data.table or tibble) |
midfield_degree |
|
midfield_term |
|
keep_text |
Deprecated |
drop_text |
Deprecated |
cip |
Deprecated |
select |
Deprecated |
midfield_x |
Deprecated |
select_add |
Deprecated |
... |
Not used for passing values; forces subsequent arguments to be referable only by name. |
sched_span |
Integer scalar |
span |
Integer scalar |
add_completion_status()is deprecated in favor of
completion_status(). Update midfieldr file names and argument names,
dropping columns not used by the function, and preserving data frame
class.
add_data_sufficiency()is deprecated in favor of
data_sufficiency(). Update midfieldr file names and argument names,
dropping columns not used by the function, and preserving data frame
class.
add_timely_term()is deprecated in favor of
timely_term(). Update midfieldr file names and argument names,
dropping columns not used by the function, and preserving data frame
class.
filter_cip()is deprecated in favor of
filter_programs(). The new function is similar but with the CIP
data frame as the first argument, enabling chained functions like those
encountered using dplyr and friends.
select_required()is deprecated in favor of
select_basic_cols(). The new functionality is similar but with
exact matching to the default column names plus preserving data
frame class.
Data frame of IDs after processing the practice data for data sufficiency and degree seeking. Provides a convenient bloc to start many of the analysis illustrated in the package articles.
baseline_mcidbaseline_mcid
data.table with 76875 rows and 1 column:
mcidCharacter. Anonymized student identifier that connects the four data tables, e.g., "MCID3111142897."
Other case-study-data:
study_observations,
study_programs,
study_results
A wrapper on base::tryCatch() for previewing an error message, if any.
catch_error(f)catch_error(f)
f |
Function with arguments expecting an error |
Does not return anything. The side effect is to output to the terminal.
# Example data frames sel_ids <- toy_student[14:18, (mcid)] s <- toy_student[mcid %chin% sel_ids, .(mcid, sex)] t <- toy_term[mcid %chin% sel_ids, .(mcid, term)] d <- toy_degree[mcid %chin% sel_ids, .(mcid, term_degree)] # No error catch_error(post_bacc_terms(t, d)) # Error, no term variable catch_error(post_bacc_terms(s, d)) # Error, missing dframe argument catch_error(post_bacc_terms()) # Error, missing degree argument catch_error(post_bacc_terms(t))# Example data frames sel_ids <- toy_student[14:18, (mcid)] s <- toy_student[mcid %chin% sel_ids, .(mcid, sex)] t <- toy_term[mcid %chin% sel_ids, .(mcid, term)] d <- toy_degree[mcid %chin% sel_ids, .(mcid, term_degree)] # No error catch_error(post_bacc_terms(t, d)) # Error, no term variable catch_error(post_bacc_terms(s, d)) # Error, missing dframe argument catch_error(post_bacc_terms()) # Error, missing degree argument catch_error(post_bacc_terms(t))
A data table based on the US National Center for Education Statistics (NCES), Integrated Postsecondary Education Data System (IPEDS), 2010 CIP. The data are codes and names for 1582 instructional programs organized on three levels: a 2-digit series, a 4-digit series, and a 6-digit series.
cipcip
A data.table with 1582 rows and 6 columns keyed by the
6-digit CIP code:
cip6nameCharacter, program name at the 6-digit level
cip6Character, 6-digit code representing "specific instructional programs" (US National Center for Education Statistics).
cip4nameCharacter, program name at the 4-digit level.
cip4Character, 4-digit code (the first 4 digits of cip6)
representing "intermediate groupings of programs that have
comparable content and objectives."
cip2nameCharacter, program name at the 2-digit level.
cip2Character, 2-digit code (the first 2 digits of cip6)
representing "the most general groupings of related programs."
The midfielddata taxonomy includes one non-IPEDS code (999999) for Undecided or Unspecified, instances in which institutions reported no program information or that students were not enrolled in a program.
https://nces.ed.gov/ipeds/cipcode/
Other cip-data:
cip2010,
fye_proxy
A data table of the 2010 Classification of Instructional Programs (CIP)
accessed in 2026 from the US National Center for Education Statistics
(NCES). Like the cip data set originally included with midfieldr,
cip2010 provides codes and names for instructional programs organized
on three levels: a 2-digit series, a 4-digit series, and a 6-digit series.
cip2010cip2010
data.table with 1849 rows and 6 columns keyed by the 6-digit CIP
code:
cip6Character, 6-digit code representing "specific instructional programs" (US National Center for Education Statistics).
cip6nameCharacter, program name at the 6-digit level
cip4Character, 4-digit code (the first 4 digits of cip6)
representing "intermediate groupings of programs that have
comparable content and objectives."
cip4nameCharacter, program name at the 4-digit level.
cip2Character, 2-digit code (the first 2 digits of cip6)
representing "the most general groupings of related programs."
cip2nameCharacter, program name at the 2-digit level.
The midfielddata taxonomy includes one non-IPEDS code (999999) for Undecided or Unspecified, instances in which institutions reported no program information or that students were not enrolled in a program.
https://nces.ed.gov/ipeds/cipcode/
Other cip-data:
cip,
fye_proxy
Determine the completion status for each student in a data frame and add columns that support the findings.
completion_status(dframe, midfield_table = degree)completion_status(dframe, midfield_table = degree)
dframe |
Data frame or data frame extension (e.g., data.table or tibble) with required variables |
midfield_table |
|
If a population has been filtered for data sufficiency, then determining every student's completion status is feasible. Completing an academic program in a timely manner means that a student completes the requirements for a degree within a set time span, typically 4, 6, or 8 years after admission depending on the definition adopted in a particular study. The term at the end of that span is the timely completion term.
If the student's degree term is no later than their timely completion term, then their completion status is "timely"; if later, their status is "late". For students with no degree, completion status is NA.
Data frame with the following properties:
Data frame class is preserved. Groups and keys are not preserved.
Row order is preserved. Rows with NA values in any
of the required variables are removed. Duplicated rows are removed.
Columns with names different from the new columns (named below) are not modified; columns with matching names are replaced. The new columns added are:
term_degree Joined from midfield_table.
completion_status Character. Possible values of "timely",
"late" and "NA".
term <- toy_term degree <- toy_degree # Start with a selected population. x <- toy_student[21:36, .(mcid, sex)] x # Add the required columns from timely_term(). x <- timely_term(x, midfield_table = term) x <- x[, .(mcid, sex, timely_term)] x # Add completion status columns. Unrelated columns (sex) are unaffected. x <- completion_status(x, midfield_table = degree) x # Repeat. New columns silently replace existing columns of the same name. y <- completion_status(x, midfield_table = degree) yterm <- toy_term degree <- toy_degree # Start with a selected population. x <- toy_student[21:36, .(mcid, sex)] x # Add the required columns from timely_term(). x <- timely_term(x, midfield_table = term) x <- x[, .(mcid, sex, timely_term)] x # Add completion status columns. Unrelated columns (sex) are unaffected. x <- completion_status(x, midfield_table = degree) x # Repeat. New columns silently replace existing columns of the same name. y <- completion_status(x, midfield_table = degree) y
Determine data sufficiency for each student in a data frame and add columns that support the findings.
data_sufficiency(dframe, midfield_table = term)data_sufficiency(dframe, midfield_table = term)
dframe |
Data frame or data frame extension (e.g., data.table or tibble) with required variables
|
midfield_table |
|
Data sufficiency is a criterion for including or excluding a student record based on the feasibility of determining their completion status given the range of data available from their institution. If determining completion status is feasible, the student record is included in the study population; if not, they must be excluded to avoid biased counts of completers and non-completers. Such biases occur at the upper and lower bounds of an institution's data range.
To apply this criterion, our heuristic labels a row "exclude-upper" when a student's timely completion term exceeds the upper limit of their institution's data range; "exclude-lower" when their initial term matches the lowest non-summer limit of the data range; and "include" otherwise. The rationale for these specific filters is explained in our data sufficiency article (see references). In most studies, the population must satisfy the data sufficiency requirement.
Data frame with the following properties:
Data frame class is preserved. Groups and keys are not preserved.
Row order is preserved. Rows with NA values in any
of the required variables are removed. Duplicated rows are removed.
Columns with names different from the new columns (named below) are not modified; columns with matching names are replaced. The new columns added are:
institution Character. Name of the institution at which a
student is enrolled in a term.
lower_limit Character. Initial term of an institution's
data range, encoded YYYYT. Extracted from midfield_table.
upper_limit Character. Final term of an institution's
data range, encoded YYYYT. Extracted from midfield_table.
data_sufficiency Character. Possible values are "include",
"exclude-lower," and "exclude-upper."
Richard Layton, Russell Long, Matthew Ohland, Marisa Orr, and Susan Lord (2026) Data sufficiency, https://midfieldr.github.io/midfieldr/articles/art-020-data-sufficiency.html
term <- toy_term # Start with a selected population. x <- toy_student[c(9:15, 342:344), .(mcid, sex)] x # Add the required columns from timely_term(). x <- timely_term(x, midfield_table = term) x <- x[, .(mcid, sex, term_i, timely_term)] x # Add data sufficiency columns. Unrelated columns (sex) are unaffected. x <- data_sufficiency(x, midfield_table = term) x # Repeat. New columns silently replace existing columns of the same name. y <- data_sufficiency(x, midfield_table = term) yterm <- toy_term # Start with a selected population. x <- toy_student[c(9:15, 342:344), .(mcid, sex)] x # Add the required columns from timely_term(). x <- timely_term(x, midfield_table = term) x <- x[, .(mcid, sex, term_i, timely_term)] x # Add data sufficiency columns. Unrelated columns (sex) are unaffected. x <- data_sufficiency(x, midfield_table = term) x # Repeat. New columns silently replace existing columns of the same name. y <- data_sufficiency(x, midfield_table = term) y
Subset a CIP data frame, retaining rows that match or partially match any string in a vector of character strings.
filter_programs(dframe, pattern, ..., negate = FALSE)filter_programs(dframe, pattern, ..., negate = FALSE)
dframe |
Data frame or data frame extension (e.g., data.table or tibble) with CIP program names and codes, e.g., the
|
pattern |
Character vector of search strings, including regular expressions. |
... |
Not used for passing values; forces subsequent arguments to be referable only by name. |
negate |
Logical (default FALSE). If TRUE, inverts the resulting Boolean vector. |
Each element of the pattern vector is matched row-wise to every
value in dframe using grepl(). If negate = FALSE (default), a
match retains the full row; if negate = TRUE, a match removes the full row.
Data frame with the following properties:
Data frame class is preserved. Groups and keys are not preserved.
Rows are a subset of the input and appear in the same order. Duplicated rows are removed.
Columns are not modified.
Groups and keys are not preserved.
# Subset using keywords filter_programs(cip, pattern = "history") # Subset using codes filter_programs(cip, pattern = "^54") # Multiple passes to narrow the results first_pass <- filter_programs(cip, "math") first_pass[, .(cip6name, cip6)] second_pass <- filter_programs(first_pass, c("bio", "educ"), negate = TRUE) second_pass[, .(cip6name, cip6)] third_pass <- filter_programs(second_pass, c("^27", "^30")) third_pass[, .(cip6name, cip6)] # Multiple passes by chaining chain_pass <- cip |> filter_programs("math") |> filter_programs(c("bio", "educ"), negate = TRUE) |> filter_programs(c("^27", "^30")) chain_pass[, .(cip6name, cip6)]# Subset using keywords filter_programs(cip, pattern = "history") # Subset using codes filter_programs(cip, pattern = "^54") # Multiple passes to narrow the results first_pass <- filter_programs(cip, "math") first_pass[, .(cip6name, cip6)] second_pass <- filter_programs(first_pass, c("bio", "educ"), negate = TRUE) second_pass[, .(cip6name, cip6)] third_pass <- filter_programs(second_pass, c("^27", "^30")) third_pass[, .(cip6name, cip6)] # Multiple passes by chaining chain_pass <- cip |> filter_programs("math") |> filter_programs(c("bio", "educ"), negate = TRUE) |> filter_programs(c("^27", "^30")) chain_pass[, .(cip6name, cip6)]
Proxies are the degree-granting engineering programs we estimate that First-Year Engineering (FYE) students would have declared had they not been required to enroll in FYE. Keyed by student ID. Proxies are provided for all students in the midfielddata practice data who enroll in FYE in their first term.
fye_proxyfye_proxy
data.table with 4623 rows and 2 columns keyed by student ID:
mcidCharacter. Anonymized student identifier that connects the four data tables, e.g., "MCID3111142897."
proxyCharacter. The 6-digit CIP code of the estimated proxy program.
The proxy variable contains 6-digit CIP codes of degree-granting engineering programs, e.g., Electrical Engineering, Mechanical Engineering, etc., that are substituted for the FYE CIP code when an analysis requires degree-granting starting programs. The most common application is a graduation rate calculation.
The estimation is based on students' first post-FYE programs and a multiple imputation suitable for categorical variables using the mice package. The predictor variables are institution, race, and sex. The estimated variable is the 6-digit CIP code of a degree-granting engineering program at their institution.
fye_proxy holds only for the practice data in midfielddata—these values
cannot be commingled with the MIDFIELD research database.
Data frame of letter grades and conventional point assignments used for computing grade point averages.
grade_scalegrade_scale
data.table with 12 rows and 2 columns:
letter_gradeCharacter, letter grades using the conventional US scale from A to F.
pointsNumerical, 4.0 scale of points assigned to letter grades.
Other scales:
sat_act_scale
A wrapper on base::str() with arguments set to not show attributes,
to not show length, and to cut the width.
look_at(x)look_at(x)
x |
Any R object. |
Does not return anything. The side effect is to output to the terminal.
# data frames look_at(cip) look_at(toy_degree) # character vectors x <- sort(unique(toy_degree$institution)) look_at(x)# data frames look_at(cip) look_at(toy_degree) # character vectors x <- sort(unique(toy_degree$institution)) look_at(x)
Condition data for Cleveland multiway charts. Two independent categorical variables are converted to factors with their levels ordered by the single quantitative response variable.
order_multiway( dframe, quantity, categories, ..., method = NULL, ratio_of = NULL )order_multiway( dframe, quantity, categories, ..., method = NULL, ratio_of = NULL )
dframe |
Data frame or data frame extension (e.g., data.table or tibble) with the following required variables: two
independent categorical variables, one quantitative response
variable, and, if |
quantity |
Character. Name of the single multiway quantitative variable. |
categories |
Character. Vector of names of the two multiway categorical variables. |
... |
Not used for passing values; forces subsequent arguments to be referable only by name. |
method |
Character. Method of ordering the levels of the categories; possible values are “median” (default) or “percent”. The median method determines medians of the quantitative column grouped by category. The percent method sums dividends and divisors by category and calculates their quotients (again, by category). |
ratio_of |
Character. Vector of names of the dividend and the divisor
that produced the quantitative variable. Required when
|
Multiway data comprise a single quantitative value (or response) for every combination of levels of two categorical variables. The ordering of the rows and panels, based on the response quantity, is crucial to the perception of effects (Cleveland, 1993).
Multiway data comprise three variables: a categorical variable of
levels; a second independent categorical variable of
levels; and a quantitative variable (or response) of
length that cross-classifies the categories,
that is, there is a value of the response for each combination of levels
of the two categorical variables. If a response value is missing, it is
assumed that a response for every combination is at least feasible.
In a multiway dot plot, one category is encoded by the panels, the second category is encoded by the rows of each panel, and the quantitative variable is encoded along identical horizontal scales.
Data frame with the following properties:
Data frame class is preserved. Groups and keys are not preserved.
Rows are preserved, though the row order may change.
Numerical variables are converted to type double. Columns
specified by categories are converted to factors and ordered.
New columns are added or replace existing columns of the same name (if any). Other columns are not modified. New columns are added as described below.
With method = median, two columns are added with names of the form
CATEGORY_median, with CATEGORY replaced with the values from the
categories argument. For example, if categories = c("program", "people"),
the two new column names would be:
program_median
people_median
With method = percent, two columns are added with names of the form
CATEGORY_QUANTITY, with CATEGORY replaced with the values from the
categories argument and QUANTITY from the quantity argument. For
example, if categories = c("program", "people") and
quantity = "grad_rate", the two new column names would be:
program_grad_rate
people_grad_rate
Groups and keys are not preserved.
Cleveland WS (1993). Visualizing Data. Hobart Press, Summit, NJ.
# Reconfigure built-in data set DT <- study_results[program == "EE" | program == "ME"] DT <- DT[race %chin% c("Asian", "Black", "Hispanic", "White")] DT[, people := paste(race, sex)] DT[, c("race", "sex") := NULL] data.table::setnames(DT, old = c("program", "graduates", "ever_enrolled", "stickiness"), new = c("prgm", "grad", "ever", "stk")) DT[] # Factor levels ordered by median mw1 <- order_multiway(DT, quantity = "stk", categories = c("prgm", "people")) data.table::setorderv(mw1, c("prgm_median", "people_median")) # The unused variables `ever` and `grad` are dropped mw1 # Levels in same increasing order as shown above levels(mw1$prgm) levels(mw1$people) # Ordering using percent method mw2 <-order_multiway(DT, quantity = "stk", categories = c("prgm", "people"), method = "percent", ratio_of = c("grad", "ever")) data.table::setorderv(mw2, c("prgm_stk", "people_stk")) # The two ratio_of variables `ever` and `grad` are retained mw2 # Levels in same increasing order as shown above levels(mw2$prgm) levels(mw2$people) # Order of factor levels depends on the method. Here, for example, # program levels are the same for median and percent methods, all.equal(levels(mw1$prgm), levels(mw2$prgm)) # but people levels do not have the same order. all.equal(levels(mw1$people), levels(mw2$people)) levels(mw1$people) levels(mw2$people)# Reconfigure built-in data set DT <- study_results[program == "EE" | program == "ME"] DT <- DT[race %chin% c("Asian", "Black", "Hispanic", "White")] DT[, people := paste(race, sex)] DT[, c("race", "sex") := NULL] data.table::setnames(DT, old = c("program", "graduates", "ever_enrolled", "stickiness"), new = c("prgm", "grad", "ever", "stk")) DT[] # Factor levels ordered by median mw1 <- order_multiway(DT, quantity = "stk", categories = c("prgm", "people")) data.table::setorderv(mw1, c("prgm_median", "people_median")) # The unused variables `ever` and `grad` are dropped mw1 # Levels in same increasing order as shown above levels(mw1$prgm) levels(mw1$people) # Ordering using percent method mw2 <-order_multiway(DT, quantity = "stk", categories = c("prgm", "people"), method = "percent", ratio_of = c("grad", "ever")) data.table::setorderv(mw2, c("prgm_stk", "people_stk")) # The two ratio_of variables `ever` and `grad` are retained mw2 # Levels in same increasing order as shown above levels(mw2$prgm) levels(mw2$people) # Order of factor levels depends on the method. Here, for example, # program levels are the same for median and percent methods, all.equal(levels(mw1$prgm), levels(mw2$prgm)) # but people levels do not have the same order. all.equal(levels(mw1$people), levels(mw2$people)) levels(mw1$people) levels(mw2$people)
For each student's term in a data frame, determine its relationship to the
student's first degree term (pre-degree, first-degree, or post-first-degree)
and add columns that support the findings. Post-first-baccalaureate terms
are typically excluded from the term, course, and degree data tables.
post_bacc_terms(dframe, midfield_table = degree)post_bacc_terms(dframe, midfield_table = degree)
dframe |
Data frame or data frame extension (e.g., data.table or tibble) with required variables |
midfield_table |
|
In a typical analysis, one is interested in a student's progress up to and including the term in which they earn their first degree or degrees. Any terms later than the first baccalaureate can usually be excluded from study.
Data frame with the following properties:
Data frame class is preserved. Groups and keys are not preserved.
Row order is preserved. Rows with NA values in any
of the required variables are removed. Duplicated rows are removed.
New columns are added or replace existing columns of the same name (if any). Other columns are not modified. The following variables are added:
first_degree_term. Character. Term of a student's first
baccalaureate, encoded YYYYT or, if no degree recorded, NA.
Joined from the term_degree variable in midfield_table.
term_cluster. Character, indicating that a term belongs
to one of three clusters: terms that are prior to ("pre-degree"),
equal to ("first-degree"), or subsequent to ("post-first-degree")
the student’s first degree term.
Groups and keys are not preserved.
# reduce number of columns term <- select_basic_cols(toy_term) degree <- select_basic_cols(toy_degree) # identify term-clusters in a 'term' table x <- post_bacc_terms(term, degree) x[, .N, by = "term_cluster"][order(-N)] x # identify term-clusters in a 'degree' table x <- post_bacc_terms(degree, degree) x[, .N, by = "term_cluster"][order(-N)] x # post-first-degree terms are usually dropped x[term_cluster != "post-first-degree"]# reduce number of columns term <- select_basic_cols(toy_term) degree <- select_basic_cols(toy_degree) # identify term-clusters in a 'term' table x <- post_bacc_terms(term, degree) x[, .N, by = "term_cluster"][order(-N)] x # identify term-clusters in a 'degree' table x <- post_bacc_terms(degree, degree) x[, .N, by = "term_cluster"][order(-N)] x # post-first-degree terms are usually dropped x[term_cluster != "post-first-degree"]
Constructs a data frame of students enrolled in First-Year Engineering (FYE) programs keyed by student ID. Conditions the data for use as an input to the mice R package for multiple imputation. Sets up three variables as predictors (institution, race/ethnicity, and sex) and one variable to be imputed (program CIP code).
prep_fye_mice( midfield_student = student, midfield_term = term, ..., fye_codes = NULL )prep_fye_mice( midfield_student = student, midfield_term = term, ..., fye_codes = NULL )
midfield_student |
|
midfield_term |
|
... |
Not used for passing values; forces subsequent arguments to be referable only by name. |
fye_codes |
Optional character vector of 6-digit CIP codes to identify FYE programs, default "140102". Codes must be 6-digit strings of numbers; regular expressions are prohibited. Non-engineering codes—those that do not start with 14—produce an error. |
At some US institutions, engineering students are required to complete a First-Year Engineering (FYE) program as a prerequisite for declaring an engineering major. Administratively, degree-granting engineering programs such as Electrical Engineering or Mechanical Engineering treat their incoming post-FYE students as their "starting" cohorts. However, when computing a metric that requires a count of starters—graduation rate, for example—FYE records must be treated with special care to avoid a miscount.
To illustrate the potential for miscounting starters, suppose we wish to calculate a Mechanical Engineering (ME) graduation rate. Students starting in ME constitute the starting pool and the fraction of that pool graduating in ME is the graduation rate. At FYE institutions, an ME program would typically define their starting pool as the post-FYE cohort entering their program. This may be the best information available, but it invariably undercounts starters by failing to account for FYE students who do not transition (post-FYE) to degree-granting engineering programs—students who may have left the institution or switched to non-engineering majors. In either case, in the absence of the FYE requirement, some of these students would have been ME starters. By neglecting these students, the count of ME starters is artificially low resulting in an ME graduation rate that is artificially high. The same is true for every degree-granting engineering discipline in an FYE institution.
Therefore, to avoid miscounting starters at FYE institutions, we have to estimate an "FYE proxy", that is, the 6-digit CIP codes of the degree-granting engineering programs that FYE students would have declared had they not been required to enroll in FYE. The purpose of 'prep_fye_mice()“ is to prepare the data for making that estimation.
After running prep_fye_mice() but before running mice(), one can edit
variables or add variables to create a custom set of predictors. The mice
package expects all predictors and the proxy variables to be factors. Do not
delete the institution variable because it ensures that a student's imputed
program is available at their institution.
In addition, ensure that the only missing values are in the proxy column. Other variables are expected to be complete (no NA values). A value of "unknown" in a predictor column, e.g., race/ethnicity or sex, is an acceptable value, not missing data. Observations with missing or unknown values in the ID or institution columns (if any) should be removed.
Data frame with the following properties:
Data frame class is preserved. Groups and keys are not preserved.
Rows: One row for every FYE student from the term input data frame.
Columns: Conditioned for later use as an input to the mice R package for multiple imputation as follows:
mcid Character, anonymized student identifier.
race Factor, race/ethnicity from the student input data
frame. An imputation predictor variable.
sex Factor, sex from the student input data
frame. An imputation predictor variable.
institution Factor, anonymized institution name from the
term data frame. An imputation predictor variable.
proxy Factor, 6-digit CIP code of a student's known,
post-FYE engineering program or NA representing missing
values to be imputed.
The function extracts all terms for all FYE students,
including those who migrate to enter Engineering after their first term,
and identifies the first post-FYE program in which they enroll, if any.
This treatment yields two possible outcomes for values returned in the
proxy column:
The student completes FYE and enrolls in an engineering major. For this outcome, we know that at the student's first opportunity, they enrolled in an engineering program of their choosing. The CIP code of that program is returned as the student's FYE proxy.
The student does not enroll post-FYE in an engineering major. Such
students have no further records in the database or switched from
Engineering to another program. For this outcome, the data provide no
information regarding what engineering program the student would have
declared originally had the institution not required them to enroll in
FYE. For these students a proxy value of NA is returned. These are the
data treated as missing values to be imputed by mice().
In cases where students enter FYE, change programs, and re-enter FYE, only the first group of FYE terms is considered. Any programs before FYE are ignored.
The resulting data frame is ready for use as input for the mice package,
with all variables except mcid returned as factors.
# Using toy data prep_fye_mice(toy_student, toy_term) # Other columns, if any, are dropped colnames(toy_student) colnames(prep_fye_mice(toy_student, toy_term)) # Optional argument permits multiple CIP codes for FYE prep_fye_mice(midfield_student = toy_student, midfield_term = toy_term, fye_codes = c("140101", "140102"))# Using toy data prep_fye_mice(toy_student, toy_term) # Other columns, if any, are dropped colnames(toy_student) colnames(prep_fye_mice(toy_student, toy_term)) # Optional argument permits multiple CIP codes for FYE prep_fye_mice(midfield_student = toy_student, midfield_term = toy_term, fye_codes = c("140101", "140102"))
Data frame for converting between ACT and SAT scores. A range of SAT scores converts to a single ACT score; an ACT score converts to a single value equivalent SAT score.
sat_act_scalesat_act_scale
data.table with 28 rows and 4 columns:
act_compNumerical, ACT composite score.
sat_lowerNumerical, total SAT, lower limit of range corresponding to the ACT composite score.
sat_equivNumerical, total SAT, value to use when converting ACT score to a single SAT score.
sat_upperNumerical, total SAT, upper limit of range corresponding to the ACT composite score.
ACT/SAT Concordance (2018) ACT Education Corp. https://www.act.org/content/dam/act/unsecured/documents/ACT-SAT-Concordance-Tables.pdf
Other scales:
grade_scale
Subset one of the four MIDFIELD data tables {student, term, course, degree}
by selecting the columns required by other midfieldr
functions.
select_basic_cols(dframe, col_pattern = NULL, ..., type = NULL)select_basic_cols(dframe, col_pattern = NULL, ..., type = NULL)
dframe |
Data frame or data frame extension (e.g., data.table or tibble) equivalent to or derived from one of the MIDFIELD
data tables: |
col_pattern |
Character vector containing strings or regular
expressions to be matched or partially matched to the column
names of |
... |
Not used for passing values; forces subsequent arguments to be referable only by name. |
type |
Character identifying the table type. Possible values are "s", "t", "c", "d", "a", or NULL (default). See Details. |
A convenience function to reduce the dimensions of a MIDFIELD data table by selecting only those columns required by other midfieldr functions or that are required to form a composite key. Particularly useful in interactive sessions when viewing the data tables at various stages of an analysis.
Several midfieldr functions require input data frames containing
specific variables (column names) such as mcid or cip6. In addition,
the MIDFIELD data tables have specific variables that act as keys
or composite keys to the information in that table. If the type argument
is NULL (default), one of the following codes is assigned to return the
column names indicated (if present):
type = "s" (student) looks for {mcid, race, sex}
type = "t" (term) looks for {mcid, term, cip6, institution, level}
type = "c" (course) looks for {mcid, term_course, abbrev, number}
type = "d" (degree) looks for {mcid, term_degree, cip6}
type = "a" looks for all the above columns
Specifying the type {s, t, c, d, a} manually in the argument overrides
the automatic selection. Additional column names can be included by using
the col_pattern argument. In all cases, unmatched search strings are
silently ignored.
Data frame with the following properties:
Data frame class is preserved. Groups and keys are not preserved.
Row order is preserved. Rows with NA values in any
of the required variables are removed. Duplicated rows are removed.
Columns are a subset of the input, appearing in the same order.
Groups and keys are not preserved.
# Basic usage select_basic_cols(toy_student[1:5]) select_basic_cols(toy_term[1:5]) select_basic_cols(toy_course[1:5]) select_basic_cols(toy_degree[1:5]) # With col_pattern for additional columns DT <- toy_student[141:146] select_basic_cols(DT, col_pattern = c("transfer", "hours_tranfer")) # Using regular expressions these_IDs <- DT$mcid DT <- toy_term[mcid %chin% these_IDs] select_basic_cols(DT, col_pattern = c("^gpa"))# Basic usage select_basic_cols(toy_student[1:5]) select_basic_cols(toy_term[1:5]) select_basic_cols(toy_course[1:5]) select_basic_cols(toy_degree[1:5]) # With col_pattern for additional columns DT <- toy_student[141:146] select_basic_cols(DT, col_pattern = c("transfer", "hours_tranfer")) # Using regular expressions these_IDs <- DT$mcid DT <- toy_term[mcid %chin% these_IDs] select_basic_cols(DT, col_pattern = c("^gpa"))
A strict version of sort() and unique() (without ...)
applied to vectors only.
sort_uniq(x, ..., na.rm = FALSE, decreasing = FALSE, na.last = FALSE)sort_uniq(x, ..., na.rm = FALSE, decreasing = FALSE, na.last = FALSE)
x |
Vector of values to be sorted with any duplicate values removed. |
... |
Not used for passing values; forces subsequent arguments to be referable only by name. |
na.rm |
Logical. Indicates if missing values (including NaN)
should be removed. Passed to |
decreasing |
Logical. Should the sort be increasing or decreasing?
Passed to |
na.last |
Logical. Position of NA values. Passed to |
A vector of unique values, sorted.
# Character vector x <- toy_student$race sort_uniq(x) # Numeric vector x <- toy_term$hours_term_attempt sort_uniq(x)# Character vector x <- toy_student$race sort_uniq(x) # Numeric vector x <- toy_term$hours_term_attempt sort_uniq(x)
Data table of post-processed observations of students ever enrolled in, and students graduating from, the four programs of the case study. Keyed by student ID. Provided for the convenience of vignette users.
study_observationsstudy_observations
data.table with 8919 rows and 5 columns.
The variables are:
mcidCharacter. Anonymized student identifier that connects the four data tables, e.g., "MCID3111142897."
raceCharacter. Race/ethnicity as self-reported by the student, e.g., "Asian", "Black", "Hispanic", etc.
sexCharacter. Sex as self-reported by the student, possible values are "Female", "Male", and "Unknown."
programCharacter. Academic program label.
blocCharacter. A group of student-level data dealt with as a unit, e.g., starters, students ever-enrolled, graduates, etc.
Starting with the case-study starting pool of students ever enrolled in the four programs of the study (Civil, Electrical, Industrial/Systems, and Mechanical Engineering), we filtered the data for data sufficiency, degree seeking, program, and timely completion.
A data frame of "ever enrolled" and a data frame of "timely graduates" were
bound using shared column names and are distinguished in the bloc variable.
This data structure facilitates grouping and summarizing by race, sex,
program, and group.
Other case-study-data:
baseline_mcid,
study_programs,
study_results
Data table of program CIP codes and labels of the four programs of the case study. Keyed by 6-digit CIPs. Provided for the convenience of vignette users.
study_programsstudy_programs
data.table with 15 rows and 2 columns. The variables are:
cip6Character. The 6-digit CIP code of the program in which a student is enrolled in a term.
programCharacter, abbreviated labels for four engineering programs. Values are "CE" (Civil Engineering), "EE" (Electrical Engineering), "ISE" (Industrial/Systems Engineering), and "ME" (Mechanical Engineering).
Starting with the midfieldr cip data set, we extracted the CIPs of the four
programs of the case study and assigned them a custom label to be used for
grouping and summarizing.
Other case-study-data:
baseline_mcid,
study_observations,
study_results
Data table of longitudinal stickiness for the four programs of the case study (Civil, Electrical, Industrial/Systems, and Mechanical Engineering) grouped by program, race/ethnicity, and sex. Provided for the convenience of vignette users.
study_resultsstudy_results
data.table with 50 rows and 6 columns:
programCharacter. Academic program label.
sexCharacter. Sex as self-reported by the student, possible values are "Female", "Male", and "Unknown."
raceCharacter. Race/ethnicity as self-reported by the student, e.g., "Asian", "Black", "Hispanic", etc.
ever_enrolledNumerical. The number of students ever enrolled in a program.
graduatesNumerical. Number of students completing a program.
stickinessNumerical. Program stickiness, the ratio of the number of graduates to the number ever enrolled, in percent.
Longitudinal stickiness is the ratio of the number of students graduating from a program to the number of students ever enrolled in the program over the time span of available data. Results are based on data that have been filtered for data sufficiency, degree seeking, and timely completion.
Other case-study-data:
baseline_mcid,
study_observations,
study_programs
Determine the timely completion term for each student in a data frame and add columns that support the findings.
timely_term(dframe, midfield_table = term, ..., sched_span = NULL, span = NULL)timely_term(dframe, midfield_table = term, ..., sched_span = NULL, span = NULL)
dframe |
Data frame or data frame extension (e.g., data.table or tibble) with required variable |
midfield_table |
|
... |
Not used for passing values; forces subsequent arguments to be referable only by name. |
sched_span |
Integer scalar (default 4), the number of years an institution officially schedules for completing a program. |
span |
Integer scalar (default 6), number of years to define timely
completion, typically 4, 6, or 8 years (100%, 150%, 200% respectively
of |
Completing an academic program in a timely manner means that a student completes the requirements for a degree within a set time span, typically 4, 6, or 8 years after admission depending on the definition adopted in a particular study. The term at the end of that span is the timely completion term.
Our heuristic assigns a time span for timely completion to every student
(default is 6 academic years). For students admitted at second-year level
or higher, the span is reduced by one academic year for each full year the
student is assumed to have completed. The adjusted span is added to their
initial term at an institution to create the timely_term value for each
observation.
Data frame with the following properties:
Data frame class is preserved. Groups and keys are not preserved.
Row order is preserved. Rows with NA values in any
of the required variables are removed. Duplicated rows are removed.
Columns with names different from the new columns (named below) are not modified; columns with matching names are replaced. The new columns added are:
term_i Character. Initial term of a student's longitudinal
record, encoded YYYYT. Extracted from midfield_table.
level_i Character. Student level (01 Freshman, 02 Sophomore,
etc.) in their initial term. Extracted from midfield_table.
adj_span Numeric. Integer span of years for timely
completion adjusted for a student's initial level.
timely_term Character. Latest term by which program completion
would be considered timely. Encoded YYYYT.
term <- toy_term # Start with a selected population. x <- toy_student[c(51:55, 346:350), .(mcid, sex)] x # Add timely term columns. Unrelated columns (sex) are unaffected. x <- timely_term(x, midfield_table = term) x # Repeat. New columns silently replace existing columns of the same name. y <- timely_term(x, midfield_table = term) yterm <- toy_term # Start with a selected population. x <- toy_student[c(51:55, 346:350), .(mcid, sex)] x # Add timely term columns. Unrelated columns (sex) are unaffected. x <- timely_term(x, midfield_table = term) x # Repeat. New columns silently replace existing columns of the same name. y <- timely_term(x, midfield_table = term) y
A subset of rows from the midfielddata course table matching the IDs in
toy_student. A small dataset for use in examples.
toy_coursetoy_course
Data frame with 8950 rows and 12 columns (data.table class).
Composite key: {mcid, term_course, abbrev, number}.
mcidCharacter. Anonymized student identifier that connects the four data tables, e.g., "MCID3111142897."
term_courseCharacter. Academic year and term,
encoded YYYYT.
abbrevCharacter. Course alphabetical identifier, e.g. "ENGR", "MATH", "ENGL."
numberCharacter. Course numeric identifier, e.g. "101", "3429."
institutionCharacter. The anonymized name of the institution the student attended in a given term, e.g., "Institution A", "Institution B", etc.
courseCharacter. Course name, e.g., "Astrophysics III", "Calculus For Social Science And Business", "Corp Financial Rprtng 1", "Environmental Sanitation II", "Fitness and Wellness", "Introductory Astronomy 2", "Our Changing Environment", etc.
sectionCharacter. Course section identifier, from one to four characters, e.g., "1", "2", "01", "14", "001", "040", "785", "H02", "R01", "300E", "888R", etc.
typeCharacter. Predominant delivery method for this section, e.g., "Blended", "Distance Education", "Face-to-Face", "Online", etc.
faculty_rankCharacter. The academic rank of the person teaching the course, e.g., "Assistant Professor", "Associate Professor", "Graduate Assistant", "Visiting Faculty", etc.
hours_courseNumeric. Number of credit-hours for successful course completion.
gradeCharacter. Course grade, e.g., "A+", "A", "A-", "B+", "I", "NG", etc.
discipline_midfieldCharacter. A variable for grouping courses by academic discipline assigned by the pre-2023 MIDFIELD data curator, e.g., "Anthropology", "Business", "Computer Science", "Engineering", "Language and Literature", "Mathematics", "Visual and Performing Arts", etc.
Other toy-data:
toy_degree,
toy_student,
toy_term
A subset of rows from the midfielddata degree table that comprises
those students from the toy_student dataset who complete a program.
A small dataset used in examples.
toy_degreetoy_degree
Data frame with 193 rows and 4 columns (data.table class).
Composite key: {mcid, term_degree, cip6}.
mcidCharacter. Anonymized student identifier that connects the four data tables, e.g., "MCID3111142897."
term_degreeCharacter. Academic year and term
in which a student completes their program, encoded YYYYT.
cip6Character. The 6-digit CIP code of the program that the student completes in this term.
institutionCharacter. The anonymized name of the institution the student attended in a given term, e.g., "Institution A", "Institution B", etc.
degreeCharacter. Type of degree awarded, e.g., "Bachelor of Arts in Geography", "Bachelor of Science in Finance," etc.
Other toy-data:
toy_course,
toy_student,
toy_term
A subset of rows from the midfielddata student table. A small dataset
for use in examples.
toy_studenttoy_student
Data frame with 351 rows and 13 columns (data.table class).
Key: {mcid}.
mcidCharacter. Anonymized student identifier that connects the four data tables, e.g., "MCID3111142897."
raceCharacter. Race/ethnicity as self-reported by the student, e.g., "Asian", "Black", "Hispanic", etc.
sexCharacter. Sex as self-reported by the student, possible values are "Female", "Male", and "Unknown."
institutionCharacter. The anonymized name of the institution the student attended in a given term, e.g., "Institution A", "Institution B", etc.
transferCharacter. Transfer status, possible values are "First-Time in College", "First-Time Transfer."
hours_transferNumeric. Number of credit hours transferred (or NA).
age_descCharacter. Age group, possible values are "25 and Older", "Under 25."
us_citizenCharacter. US citizenship, possible values are "No", "Yes."
home_zipCharacter. Home ZIP code (or NA), e.g., "02056", "20170", "51301", "80129", etc.
high_schoolCharacter. Code for the last high school attended before admission (or NA), e.g., "060075", "210512", "431800", "502195", etc.
sat_mathNumeric. SAT mathematics test score (or NA).
sat_verbalNumeric. SAT reading test score (or NA).
act_compNumeric. ACT composite test score (or NA).
Other toy-data:
toy_course,
toy_degree,
toy_term
A subset of rows from the midfielddata term table matching the IDs in
toy_student. A small dataset for use in examples.
toy_termtoy_term
Data frame with 1821 rows and 13 columns (data.table class).
Composite key: {mcid, term}.
mcidCharacter. Anonymized student identifier that connects the four data tables, e.g., "MCID3111142897."
termCharacter. Academic year and term the student
attended, encoded YYYYT.
cip6Character. The 6-digit CIP code of the program in which a student is enrolled in this term.
institutionCharacter. The anonymized name of the institution the student attended in a given term, e.g., "Institution A", "Institution B", etc.
levelCharacter. Academic level of the student at the end of this term, e.g., "01 First-Year", "02-Second Year", etc.
standingCharacter. Academic standing during the reported term, e.g., "Good Standing", "Academic Warning", etc.
coopCharacter. Cooperative education term, possible values are "Yes", "No."
hours_termNumeric. Credit hours earned in the term.
hours_term_attemptNumeric. Credit hours attempted in the term.
hours_cumulNumeric, cumulative credit hours earned.
hours_cumul_attemptNumeric. Cumulative credit hours attempted.
gpa_termNumeric. Term grade point average.
gpa_cumulNumeric. Cumulative grade point average.
Other toy-data:
toy_course,
toy_degree,
toy_student