# Introduction tudent retention is an indicator of academic performance and enrolment management of the university. Increasing student retention or persistence is a long term goal in all academic institution. High rates of student attrition have been a Author ? : Chittagong, Bangladesh. E-mail : linkon_cse_cu@yahoo.com Author ? : Chittagong, Bangladesh. E-mail : bubly.csecu@gmail.com concern for the past several years at the Chittagong University Institute of Computer Science and Engineering. Level of retention rate goes higher by various reasons and one the most important reasons are government funding, scholarship in the tertiary education environment. There are both academic and administrative staff are under pressure to come up with strategies that could increase retention rates on their courses and programs. The lowest student retention rates at all institutions of higher education are first-year students, who are at greatest risk of dropping out in the first term or semester of study or not completing their programs or degree. Therefore most retention studies address the retention of first-year students. Consequently, the early identification of vulnerable students who are prone to drop their courses is crucial for the success of any retention strategy. This would allow educational institutions to undertake timely and pro-active measures. Once identified, these at risk students can be then targeted with academic and administrative support to increase their chance of staying on the course. Data from 2004 to 2010, covering over 200 enrolled students stored in the Computer Science and Engineering was used to perform a quantitative analysis of study outcome. There were three main types of objectives conducted in this study. The first approach is descriptive which is concerned with the nature of the dataset such as the frequency table and the relationship between the attributes obtained using cross tabulation analysis (contingency tables). In addition, feature selection is conducted to determine the importance of the prediction variables for modelling study outcome. The third type of data mining approach, i.e. predictive data mining is conducted by using two different types of classification trees. The classification tree models have some advantages. Secondly, the classification tree models are non-parametric and can capture nonlinear relationships and complex interactions between predictors and dependent variable. We decided not to use other data mining techniques such as neural networks and support vector machines even though in some cases they could achieve higher accuracy, because their structure is not transparent and usually described as a black box. It is also difficult to explain their results and how they work to a user who would like S value) for best predicators of dependent variable. Then we have classify the data using the latest data mining technique Support Vector Machines(SVM).SVM helped the data set to be properly design and manipulated. After being processed data, we used the MATH LAB for depiction of resultant data into figure. After being separation of factors we have had examined by using data mining techniques Classification and Regression Tree (CART) and Bayes theorem using knowledge base. Proposition logic is used for designing knowledge base. Bayes theorem will perform the prediction by collecting the information from knowledge Base. Here we have considered most important factors to classify the successful students over unsuccessful students are gender, financial condition and dropping year. We also consider the sociodemographic variables such as age, gender, ethnicity, education, work status, and disability and study environment that may inflounce persistence or academic success of students at university level. We have collected real data from Chittagong University Bangladesh from numerous students. Finally, by mining the data, the most important factors for student success and a profile of the typical successful and unsuccessful students are identified. # Global # C to apply them to a new set of data. Finally, a comparison between these models was conducted to determine the best model for the dataset. The population of this study was entering CSE in the firstyear from 2004 through 2009. These were full-time degree-seeking students. Entering in the first-year session were excluded from the study for a few reasons. The independent or predictor variables fell into three main categories. These were the students' personal information, high school background. The personal information recorded for each student was: ? Ethnicity(Bengali, Marma, Chakma and Others) ? Gender ? Age ? Financial Status ? Work Status ? Disability II. # Collected statistical data As part of the data-understanding phase we carried out the data on the table 1 and table 2. The Table 2 reports the results. Based on the results shown majority of Information Systems students are female (over 38%). However, percentage of female students who successfully complete the course are higher (41%) which suggests that female students are more likely to [3] pass the course than their male counterpart. When it comes to age over 26% of students are above 24. This age group is also more likely to fail the course because their percentage of students who failed the course in this age group (11.9%) is higher than their overall participation in the student population (26.2%). Statistical data on 42 students: Students with it are more likely to fail than those without it. There are huge differences in percentage of students who successfully completed [4] the course depending on their ethnic origin. A substantial number of students (over 55%) have financial support more vulnerable than the other two categories in this variable. # III. # Support vector machines Support Vector Machine (SVM) is one of the latest clustering techniques which enables machine learning concepts to amplify predictive accuracy in the case of axiomatically diverting data those are not fit properly. It uses inference space of linear functions in a high amplitude feature space, trained with a learning algorithm. It works by finding a hyper plane that linearly separates the training points, in a way such that each resulting subspace contains only points which are very similar. First and foremost idea behind Support Vector Machines (SVMs) is that it constituted by set of similar supervised learning. An unknown tuple is labeled with the group of the points that fall in the same subspace as the tuple. Earlier SVM was used for Natural Image processing System (NIPS) but now it becomes very popular is an active part of the machine learning research around the world. It is also being used for pattern classification and regression based applications. The foundations of Support Vector Machines (SVM) have been developed by V.Vapnik. SVM is very effective in various data and information classification process. An expert should bear in mind two important factors for implementing SVM, these two factors or techniques are mathematical programming and kernel functions. Kernel methods leads or portrayal data into colossal amplitude margins in the anticipation that in this colossal amplitude margin the data could become more easily separated or better structured. Mathematical Programming refers the conception of the Linear programming for the best fit of Hyper plane. The word programming means to plans or make a time table for regular work. Integer Linear programming (ILP) which is the part of linear programming is very useful analytical and engineering tools to get an optimal solution .The parameters are found by solving a quadratic programming problem with linear equality and inequality constraints; rather than by solving a nonconvex, unconstrained optimization problem. The flexibility of kernel functions allows the SVM to search a wide variety of hypothesis spaces. The for-most reasons of using SVM are to select the proper Support Vectors for the data classification. The figure 1 shows a graphical view of Support Vectors selection of the process. All hypothesis space help to identify the Maximum Margin Hyper plane (MMH) which enables to classify the best and almost correct data the following figure shows the process of SVMs selection from large amount of SVMs. We can calculate the weight boundary maximum margin by using the following equation: Another interesting question is why maximum margin? There are some good explanations which include better empirical performance. Another reason is that even if we've made a small error in the location of the boundary this gives us least chance of causing a misclassification. The other advantage would be avoiding local minima and better classification. The goals of SVM are separating the data with hyper plane and extend this to non-linear boundaries using kernel trick [8] [11]. For calculating the SVM we see that the goal is to correctly classify all the data. For mathematical calculations we have, In this equation x is a vector point and w is weight and is also a vector. So to separate the data [a] should always be greater than zero. Among all possible hyper planes, SVM selects the one where the distance of hyper plane is as large as possible. If the training data is good and every test vector is located in radius r from training vector. Now if the chosen hyper plane is located at the farthest possible from the data [12]. This desired hyper plane which maximizes the margin also bisects the lines between closest points on convex hull of the two datasets. Thus we have [a], [b] & [c]. # Classification and regression tree (cart) Classification and Regression Tree (CART) has many advantages over classification methods. It is naturally non-parametric. CART can handle [5][6] numerical data that are highly skewed. It eliminates analyst time, which would otherwise be spent determining whether variables are normally distributed. CART identifies "splitting" variables based on a complete search of all possibilities. CART is able to search all possible variable splitters, even in problems with many hundreds of possible predictors as well as dealing missing variable. Finally, CART trees are moderately simple and non-statisticians. The purpose of an analysis based on classification tree is to find out the factors that contribute the separation of successful and unsuccessful students among the all recorded data. When the classification tree is generated it is possible to calculate the probability of each student's outcome. If once the classification tree is formed, it is possible to predict the study outcome for newly enrolled student. In each tree node the percentages for successful and unsuccessful students is given and also absolute size of the node. The variable names above the node are the predictor's criteria that make the split for the node according to classification and regression tree. Each node is split according to predictor criteria. The searching is stops when the split with the largest improvement in goodness of fit. Possible predictor variables in the dataset were included in the classification tree in splits process, which is detected in feature selection criteria. We used two stopping criteria in the training process: 1. A maximum tree depth has been proceeding into 3 levels for CHAID tree. 2. A maximum tree depth has been proceeding into 4 levels for CART tree. Lastly for each classification tree we have assigned different costs to the classification outcomes i.e.: classification matrix. It is one of the processes of increasing the correctly classified student's outcomes. We categories the student's outcome according to Pass and Fail in the academic courses In this research we explore the concepts and technique of SVM to classify the data collected in our experiments. We have approximately collected twelve hundreds (1200) data from University of Chittagong where about thirty (30000) thousands students are studying. To assess these large amounts of data we have found that SVM is very efficient and exact technique in our proceedings. By imposing the SVM, we have mapped the data to meaning full forty two (42) data which are shown in the figure 3 and 4.In figure 3 we have depicts that the reasons for the age where mainly focused on the pivotal age of greater or less than twenty four. Fig. 3 : Data classification for age group using SVM From the figure above we can easily measure the Maximum Margin Hyper plane (MMH).At MMH the resultant outcome of age group using SVM is determined. We have also used the MATHLAB to accelerate the accuracy of the implementation. In the same process we have had accomplished our design and implementation for the financial support data using the methodology of SVM. Step1: First insert the observed value in each cell of observable table. Inserted value collected from record. Step 2: Calculate expected value for every cell of the describing table . Step 3: calculating chi value for every cell using the following formula: ?² = (observed value-expected value)²/expected value Step 4: calculate total chi -value for domain using the following formula (observed value-expected value)²/expected value Step 5: calculating degree of freedom using following rule Degree of freedom, df = (No.of.rows-1)*(No.of.columns-1) Step 6: calculate p-value (probability value) using following method in Ms Excel P-value=CHIDIS (Chi value, df) Step 1: consider the domain is financial support in which category1=Yes category2=No, Option1=Pass and Option2=Fail. The value of every cell collects from database. Step 2: calculating expected value for each cell using describing formula Step 3: calculating chi value for every cell using the describing formula: Step 4: calculate total chi -value for domain ?²= 0.283+0.631+0.343+0.764=2.021 Overall chi-value for every domain in Pass and Fail Category: Table 3 : Individual Chi-value for each category Step 5: calculating degree of freedom using following the rule Degree of freedom df = (2-1)*(2-1) = 1 Step 6: calculate p-value (probability value) using in Ms Excels P-value =CHIDIS (2.021, 1) = 0.155 Factors selection is an important process to assess the prediction of dropout of the students. The prediction has relate the variable that determines the rate of success The number of predictor variables is not so large and we don't have to select the subset of variables for further analysis which is the main purpose of applying feature selection to data. However, feature selection could be also used as a pre-processor for predictive data mining to rank predictors according to the strength of their relationship with dependent or outcome variable. During the factors selection process no specific form of relationship, neither linear nor nonlinear, is assumed. The outcome of the factors selection would be a rank list of predictors according to their importance for further analysis of the dependent variable with the other methods for regression and classification. Here the figure below shows the relative outcome of the predictor's value. In all three cases, i.e. for all three definitions of the dependent variable, if the top 4 variables are selected, we get the same list of predictors. Therefore we can conclude that the list of important predictors is quite robust to changes in the study outcome definition. We may proceed into the next step using the top 4 variables: 1. Financial Support 2. Age Group 3. Gender 4. Disabilities From Table 4, P-values from the last column only the first three chi-square values are significant at 10% level. Though the results of the feature selection suggested continuing analysis with only the subset of predictors, which includes Financial Support, Age group and Gender, we have included all available predictors in our classification tree analysis. We follow an advice given in Luan & Zhao (2006) who suggested that even though some variables may have little significance to the overall prediction outcome, they can be essential to a specific record. In our work we have described a simple method of probabilistic inference that is, the computation from observed evidence of posterior probabilities for query propositions. We have used the joint probability as the knowledge base from which answer to all question may be derived. We have had built the knowledge base by considering two Boolean variables. The table 7 is an example of two valued propositional logic which is the bases of knowledge base representation: Table 7: Concepts of propositional logic to design a Knowledge Base using the proposition of Boolean events A, B and C. Based on table 7, we have designed the knowledge base (Joint probability distribution) for our research activity. Here we have considered those events which have true (one or 1) Boolean values. Table 8 is an example of knowledge base for events A, B and C: Bayes' theorem and conditional probability are opposite to each other. Given two dependent events A and B. The conditional probability of P (A and B) or P (B/A) will be P (A and B)/P (A). Related to this formula a rule is developed by the English Presbyterian minister Thomas Bayes (1702-61).According to the Bayes rule it is possible to determine the various probabilities of the first event given the outcome of the second event in a sequence of two events. The conditional probability: (1) The equation (1) will help to find out the probabilities of B after being occurrences of the A. we get the Bayes' theorem for these two events as follows: (2) If there are more events like A1, A2, and B1, B2.In this case the Bayes theorem to determine the probability of A1 based on B1will be as follows: Consideration Classification and Regression Tree (CART): For Node 4: IF Financial Support="Yes" AND Age<=24 THEN "PASS"= (18/23)*(9/15) =0.49 Now applying the Bayes theorem on table 5 we have got the following outcomes: If one student pass based on his financial condition is "Yes" and age<=24 then P (Pass | Financial condition="Yes" ^age<=24) = P (Pass ^Financial condition="Yes" ^age<=24)/ P (Financial condition="Yes" ^age<=24) P (Pass ^Financial condition="Yes" age<=24) =0.279 P (Financial condition="Yes" ^ age<=24) =0.125+0.279 = 0.404 P (Pass | Financial condition="Yes" ^age<=24) =0.125/0.404=0.31 The total resultant of Bayes Theorem and CART of all data considering financial condition and age group we have got the following table 7: ![Journal of Computer Science and TechnologyVolume XII Issue XV Version I](image-2.png "") 1![Figure 1 : Representation of Support Vectors](image-3.png "GlobalFigure 1 :") 2![Figure 2 : Representation of Hyper planes Distance of closest point on hyper plane to origin can be found by maximizing the x as x is on the hyper plane. Similarly for the other side points we have a similar scenario. Thus solving and subtracting the two distances we get the summed distance from the separating hyper plane to nearest points. Maximum Margin = M = 2 / ||w|| IV.](image-4.png "Figure 2 :") ![If Y i = +1; [b] If Y i = -1; wx i + b ? 1 [c] For all i; y i (w i + b) ? 1](image-5.png "") 4![Fig. 4 : SVM to classify the financial support data Now, we have calculated the chi square values of collected data. The procedures of chi square values are given bellow:Step1: First insert the observed value in each cell of observable table. Inserted value collected from record.](image-6.png "Fig. 4 :") 5![Figure 5 : Importance plot for predictors Results of factors selection has presented in Figure 3 on the importance plot and inTable 1. The top three predictors for the study outcome are financial support of students, disabilities, age group and gender in which they are study.](image-7.png "Figure 5 :") 1 2 4FinancialPass Fail TotalSupportYes18523No11819Total29134259Financial Support Yes No Total Financial Support YesPass 23*29/42=15.88 23*13/42=7.12 23 Fail Total 19*29/42=13.12 19*13/42=5.88 19 29 13 42 Pass Fail ?²=(18-?²=(5-7.12)²/7.12Volume XII Issue XV Version I D D D D )15.88)²/15.88= 0.631(No Domain Name Financial Support Age group Gender= 0.283 ?²=(11-13.12)²/13.88 = 0.343 Possible category Yes No Age>24 Age<24 MalePass 0.283 0.343 0.337 0.120 0.050?²=(8 -5.88)²/5.88 = 0.764 Fail 0.631 0.764 0.753 0.267 0.112Global Journal of Computer Science and TechnologyDisabilitiesFemale Yes Female0.082 0 0.200.182 1.54 0.040Domain Name Financial SupportChi-value 2.021P-value .155Disabilities1.6.206Age Group1.477.224Gender.426.514 5Table 6 : Fully join probability distribution(Knowledge base)Where0.125+0.044+0.103+0.037+0.279+0.099+0.231+0.082=1 © 2012 Global Journals Inc. (US) 1 © 2012 Global Journals Inc. (US) This study examines the background information from enrolment data that impacts upon the study outcome of Information Systems students at the Department of Computer Science & Engineering. Based on results from feature selection (Figure 5 and Table 4), the CART trees presentation it was found that the most important factors that help separate successful from unsuccessful students are financial support , age group and gender. Demographic data such as gender and age though significantly related to the study outcome, according to the feature selection result. Based on results from table 5 and 6 by implementing the knowledge of propositional knowledge base and Bayes theorem based on knowledge base to predict the academic success. Better result is found from using Bayes Network. This study is limited in three main ways that future research can perhaps address. Firstly, this research is based on background information only. Secondly, we used a dichotomous variable for the study outcome with only two categories: pass and fail. Thirdly, from a methodological point of view an alternative to a classification tree should be considered. The prime candidates to be used with this data set are logistic regression and neural networks. * Mining student data using decision trees QAAl-Radaideh EMAl-Shawakfa MIAl-Najjar the Proceedings of the 2006 International Arab Conference on Information Technology 2006. 2006 * New Dropout Prediction for Intelligent System SarwarMd LinkonKamal Chowdhury International Journal of Computer Applications 16 42 March 2012 Sonia Farhana Nimmy * A conceptual model of non traditional undergraduate student attrition JPBean BSMetzner Review of Educational Research 55 1985 * An econometric analysis of student withdrawal and progress-sion in post-reform Italian universities. Centro Ricerche Economiche Nord Sud -CRENoS Working Paper GBoero TLaureti RNaylor 2005. 2005/04 * Using data mining to predict secondary school student performance PCortez ASilva the Proceedings of 5th Annual Future Business Technology Conference Porto, Portugal 2008 * Predicting student drop out: A case study GWDekker MPechenizkiy JMVleeshouwers 2009 * Proceedings of the 2nd International Conference on Educational Data Mining (EDM'09) the 2nd International Conference on Educational Data Mining (EDM'09)Cordoba, Spain July 1-3 * The elements of statistical learning: Data mining, inference and prediction THastie RTibshirani JFriedman 2009 Springer New York 2nd ed. * Investigation of the role of pre-and post admission variables in undergraduate insti-Mutational persistence, using a Markov student flow model JHan MKamber Data mining: Concepts and techniques Amsterdam; USA PhD Dissertation 2006. 2006 North Carolina State Uni-versity 2nd ed. * Future time orientation predicts academic engagement among first-year university students LHorstmanshof CZimitat British Journal of Educational Psychology 77 3 2007 * A longitudinal approach to assessing attrition behavior among first-generation stu-dents: Time-varying effects of precollege characteristics TTIshitani Research in Higher Education 44 4 2003 * Studying attrition and degree completion behavior among first-generation college students in the United States TTIshitani Journal of Higher Education 77 5 2006 * Understanding dropout of adult learners in e-learning. PhD Dissertation, the University of JJun 2005 Georgia, USA * Open learning courses for adults: A model of student progress DKember 1995 Education Technology Englewood Cliffs, NJ * Practicing data mining for enrolment management and beyond. New Di-reactions for Institutional Research JLuan C-MZhao 2006 31 * Predicting the retention of university students PMurtaugh LBurns JSchuster Research in Higher Education 40 3 1999 * Enrollment prediction models using data mining ANandeshwar SChaudhari 2009. 2010 * Handbook of statistical analysis and data mining applications RNisbet JElder GMiner 2009 Elsevier Amsterdam * Predicting successful college experiences: Evi-dence from a first year retention program KNoble NTFlynn JDLee DHilton Journal of College Student Retention: Research, Theory & Practice 9 1 2007 * A test and reconceptualization of a theoretical model of college withdrawal in a commuter institution setting ETPascarella PBDuby BKIverson Sociology of Education 56 1983 * First-generation college students: Are they at greater risk for attrition than their peers? PAPratt CTSkaggs Research in Rural Education 6 1 1989 * Student variables that predict retention: Recent research and new developments RDReason NASPA Journal 40 4 2003 * Data mining with decision trees -Theory and applications LRokach OMaimon 2008 World Scientific Publishing New Jersey * A data-mining approach to differentiate predictors of retention CHYu SDigangi AJannasch-Pennell WLo CKaprolet the Proceedings of the Educause Southwest Conference Austin, Texas, USA 2007 * Educational data mining: A survey from CRomero SVentura Expert Systems with Applications 33 2007. 1995 to 2005 * Predicting student success in open and distance learning OSimpson Open Learning 2006 21 * Uncovering hidden information within university's student enrolment data using data mining FSiraj MAAbdoulha MASAUM Journal of Computing 1 2 2009 * An examination of the impact of first-year seminars on correlates of college stu-dent retention TLStrayhorn Journal of the First-Year Experience & Students in Transition 21 1 2009 * Predicting persistence of urban commuter campus students utilizing student background characteristics from enrollment data JTharp Community College Journal of Research and Practice 22 1998 * Artificial Intelligence A modern Approach by Stuart Russell and Peter Norvig * The state of educational data mining in 2009: A review and future visions RS J DBaker KYacef Journal of Educational Data Mining 1 2009