Analysis of admission data and student performance

In this document, the admission data from IfI (UZH) is analyzed and visualized. In addition, the student performance is evaluated, and a recommendation for the selection of students is given, which is based on the results of this analysis.

1 Data preparation

1.1 Available data

1.2 Data preprocessing & cleaning

2 Preliminaries

2.1 Interactive visualizations

Most of the visualizations are interactive, and you can zoom in and out, or hover over plot elements to obtain more information. Zooming in might be required to see all the details of the plot as some of the plots are quite large. To zoom in, simply select the area you want to zoom in on with your mouse by clicking and dragging the mouse to create a rectangle. To zoom out again, double-click on the plot.

2.2 Box plots

In some cases we present the distribution of the data as box plots.

Figure 1 explains how a box plot is constructed. An exemplary distribution is shown in blue and the corresponding box plot below. The box shows the interquartile range ($IQR$) which ranges from the first $Q1$ to the third quartile $Q3$ of the data distribution. The line in the middle of the box marks the value of the median. The upper fence ranges from the third quartile $Q3$ to the value which equals $Q3 + 1.5 \cdot IQR$. The lower fence is defined in the same way as $Q1 - 1.5 \cdot IQR$. Values that fall outside the range $[Q1 - 1.5 \cdot IQR, Q3 + 1.5 \cdot IQR]$ are considered to be outlier and displayed as dots.

3 Analysis of admission data

3.1 Breakdown of admission numbers

Figure 2 shows the total number of applicants per semester and major/minor. The applications are broken down into the categories "Accepted - started", "Accepted - not started" and "Rejected". Hover over the bars to see the percentage breakdown.

3.1.1 Breakdown by major

The following plots, Figure 3 to Figure 6, show the breakdown of the admission numbers by semester.

In the following interactive plot (Figure 7), you can select majors and indicators to analyze trends in the admission rates. Initially, three time series are selected (the trends for the major DS). Click on the items in the legend to select or deselect items.

Figure 8 shows in applications by year. The spring (FS) and fall semester (HS) are aggregated. A breakdown by major is shown.

3.1.2 Breakdown by minor

Below, in Figure 9, a breakdown of the admission numbers is shown for the minor programs. All applications from FS20 to HS23 are considered.

In Figure 10 to Figure 13, the previous analysis is repeated for the minor programs by semester.

3.2 Sources of students by major

The Sankey charts below show where students of the different major programs come from. If students are new to the University of Zurich, then the region of the university is displayed from where the students obtained their bachelor's degrees. If the students have been at UZH before and changed their major program, then the previous major program is displayed.

For example, if a student joined the UZH in FS20 from another university in Switzerland to study SOSY, then the source of this student is "Switzerland" in FS20 and the target is SOSY. If the same student then changes the major program from SOSY to IS in HS22, then the source is displayed as SOSY and the target is IS.

There is no information on the provenance of the students who migrated but started their studies before FS20. If those students changed their major program, then the source is displayed as "UZH major (unknown)".

Figure 14 shows the sources of students and the targets, i.e. the selected major programs, in aggregate for all semesters from FS20 to HS23.

In the subsequent Figures 15 to 22 the figures are broken down by semester.

3.2.1 Breakdown by semester

3.3 University region of students who were accepted and did not start

Below, the bar chart in Figure 23 shows where students come from who were accepted by UZH but did not start their studies.

3.4 Limitations of the data

Unfortunately, the data on the university ranking of the university from which the applicants obtained their bachelor's degree is not complete. The table below show that most data is missing for Swiss students and for students from EU/EFTA.

Thus, no correlation analysis between the university ranking and the admission rate can be performed for students from Switzerland and EU/EFTA. However, an analysis is performed that investigates whether the university ranking is correlated with the grades of the students from Extra-EU/EFTA countries. Only Extra-EU/EFTA students for which the university ranking is available are considered in this analysis.

4 Analysis of student performance

In the following analyses, each sample represents one graded course for a student.

Note, the grade 1 was not considered in the following analyses as this usually indicates that the student did not attend the exam or dropped out of the course after the enrollment deadline.

4.1 Performance in courses which are additional requirements

Some students are required to take additional courses to be admitted to the master program, referred to as "additional requirements". The following plots show the performance of students in these courses. In the subsequent analyses, the grades obtained in additional requirements are not considered. However, even when including the grades of the additional requirements, the results and conclusions of the following analyses do not change.

EU/EFTA and Extra EU/EFTA tend to take a similar number of additional requirements. Students from Switzerland tend to take fewer additional requirements in comparison to students from EU/EFTA and Extra EU/EFTA. While Swiss students perform as good in the additional requirements as they do in the other courses, students from EU/EFTA and Extra EU/EFTA perform worse in the additional requirements than in the other courses that they take, as displayed in Figure 24.

The table below shows the number of grades obtained in courses which are either additional requirements or not. The table is broken down by the university region.

4.2 Feature correlation with grades

The table below shows the correlation between features and the grades that students achieved in their courses. The correlation is computed using the Pearson correlation coefficient $r$. The correlation coefficient ranges from -1 to 1. A value of 1 indicates a perfect positive linear relationship between two variables. A value of -1 indicates a perfect negative linear relationship between two variables. A value of 0 indicates that there is no linear relationship between the two variables. The p-value indicates the probability that the observed correlation is a product of chance and that the true correlation is actually zero. It is customary to reject the null hypothesis (the variables are not correlated, i.e. the Pearson correlation coefficient is zero) if the p-value is smaller than 0.05. In this case, the correlation is considered to be statistically significant.

A positive correlation indicates that students with higher values of the feature tend to achieve higher grades. A negative correlation indicates that students with higher values of the feature tend to achieve lower grades. Note, categorical features are one-hot encoded. For example, the feature "university_region_Schweiz" takes value 1 if the student obtained their bachelor's degree at a Swiss university and 0 otherwise. Thus, a positive correlation indicates that students who obtained their bachelor's degree at a Swiss university tend to achieve higher grades.

The p-value is the probability that the null hypothesis is true.

p-value calculation:

For more details see the section "Testing using Student's t-distribution" in Wikipedia - Pearson correlation coefficient.

Interpretation of correlation coefficients

The table above shows the correlation of each feature with the grades. Some features are correlated. For example, DS and POC have disproportionally many students from Extra EU/EFTA countries who tend to perform worse.

In addition, only students from EU/EFTA and Switzerland obtained their bachelor's degree at an institution that is defined as "FH". Thus, the feature "FH" captures the university region implicitly and the positive correlation between having studied at a FH and the grades can be attributed to the university region as well.

Below, one can find the breakdown of students from institutions of type "Uni" and "FH" by region.

Taken those inter-feature correlations into account, the following important features can be identified:

The educational background is also correlated with the region. For example, 50% of the Swiss students do not have a background in computer science / informatics. However, the imbalance is not as significant as for other features. Thus, the educational background is considered to be an indicator of student performance.

4.3 Performance of students by region of university from which bachelor degree was obtained

Figure 25 shows the distribution of the grades by the region of the university from which the bachelor's degree was obtained. For instance, we can read the Extra EU/EFTA plot as follows: 25% of the students from Extra EU/EFTA achieved a grade of 4.0 or below. 50% of the students achieved a grade of 4.25 to 5.25. 75% of the students achieved a grade of 5.25 or below. The median grade is 4.75. The mean grade is 4.7. The top 25% achieved a grade above 5.25.

4.4 Performance of students by major

Below in Figure 26, the student performance by major is displayed. Note, as pointed out before, the performance by major is influenced by the composition of the student body. For example, the major DS has a high proportion of students from Extra EU/EFTA countries who tend to perform slightly worse.

For instance, we can read the Data Science major plot as follows: 25% of the Data Science major students achieved a grade of 4.0 or below. 50% of the students achieved a grade of 4.25 to 5.25. 75% of the students achieved a grade of 5.25 or below. The median grade is 4.75. The mean grade is 4.7. The top 25% achieved a grade above 5.25.

Above, the distribution of grades by major allow for the assumption that DS and POC major students perform slightly worse than the other students. However, a previous analysis shows that students from Extra EU/EFTA, tend to perform slightly worse. Both majors, DS and POC, have overproportionally many Extra EU/EFTA students (see table below). Thus, the lower performance of DS and POC students can be explained by another variable, i.e. the university region.

The assumption is confirmed by drawing a sample of grades for each major so that each university region is equally represented in the sample. The box plot in Figure 26 shows the distribution of grades for each major. The distribution of grades for DS and POC students is now similar to the distribution of grades for students in other majors. Thus, the lower performance of students with DS or POC major can be explained by the fact that they tend to come from universities outside of EU/EFTA.

Sampling:

This experiment was repeated 10 times, with a very similar result. Figure 27 shows one instance of the experiment.

4.5 Student performance of IfI major students by educational background

Figure 28 shows that students who have no educational background (fachfremd) in informatics performed as well as students who have a bachelor's degree in informatics (facheigen). The best performing group are those students whose background is related to informatics (teilweise fachfremd).

4.6 Student performance and university ranking

Note, until 2023, the admission of students was not restricted by the university ranking. Therefore, the data set contains students from universities ranked worse than 1000.

We would like to investigate the correlation between student performance and university ranking. For this, we chose two models, an Ordinary Least Squares (OLS) model of degree 1 and of degree 2. The model predicts the grade given the university ranking.

For the OLS model of degree 1, the data is not standardized to allow for a better interpretation of the model coefficients. It is important to note that a numerically smaller value for the rank means that the university is ranked higher; likewise a numerically larger value for the rank means that the university is ranked lower. This is important when interpreting the coefficients of the OLS model.

As previously noted, the data on the university rankings is insufficient for students from Switzerland and from EU/EFTA. Thus, the following analysis is only based on students from Extra EU/EFTA countries.

The analysis of the OLS regression model shows no statistically significant correlation between the university ranking and the student performance.

The coefficient of the OLS model represents the relationship between the university ranking and the students' grades. If students from low ranked universities would perform worse than students from high ranked universities, then the coefficient would be negative. (If the numerical value of the rank gets smaller, the grade gets larger; and vice versa). However, the OLS analysis shows that the coefficient is very close to zero ($-9.5 \cdot 10^{-6}$). In addition, the coefficient is not statistically significant ($\text{p-value} > 0.5$). This indicates that there is no relationship between the university ranking and the students' performance. The very small $R^2$ value of $1.4 \cdot 10^{-4}$ indicates that the university ranking does not explain the variance in the students' grades.

Hover over the regression line in Figure 29 to see the parameters of the OLS model. In addition, the text shows the predicted grade for a given university ranking.

Conclusion

While it is true that students from Extra-EU/EFTA tend to perform worse than their Swiss and EU/EFTA peers, the university ranking is not a suitable indicator to select students from those countries. The previous analysis shows that the university ranking does not lead to the selection of high performing students. Instead, other indicators such as the student performance in their bachelor's degree (e.g. ranking in graduating class: top 10%, top 25%, top 50%) should be considered. Alternatively, tests like the GRE could be used to select students from Extra-EU/EFTA countries.

Alternative models, for example a polynomial regression of degree 2 confirm the hypothesis that the university ranking is not correlated with the student performance, as presented below in Figure 30.

4.7 Student performance in core electives

Students tend to perform better in their core electives compared to other courses, as shown by the box plot below. This pattern also holds when the performance in the core electives is compared across majors. The slightly lower performance of DS and POC students can be attributed to the fact that those majors have an overproportional number of students from Extra-EU/EFTA countries, as shown in the previous analysis. See Figure 31 and 32 for details.

4.8 Study duration

Below in Figure 33, the distribution of study duration is shown for each major. The study duration is defined as the number of semesters studied at UZH. Thus, if a student changes the major program, the number of semesters will include the semesters studied in the previous major program and the semesters studied in the new major program. Note, many students need to complete additional requirements which prolongs the study duration.

5 Proposal

As outlined in this analysis, the university ranking is not a suitable indicator to select students from Extra-EU/EFTA countries. Instead, the student performance in their bachelor's degree (e.g. ranking in graduating class) should be considered. Alternatively, test like the GRE could be used to select students from Extra-EU/EFTA countries.

One has to consider that top students receive offers from multiple universities and that fewer of the top students will accept an offer from UZH. Therefore, a multi-stage admission process can be useful. First, the top applicants receive an offer, and the lower ranked applicants are notified that they are placed on a waiting list. Spots which are not filled by top applicants are filled from the waiting list. This process is repeated until all spots are filled.