forrja: In order to evaluate the performance of proposed hybrid schemes,

 In order to evaluate the performance of proposed hybrid schemes,

1- Experiments and results 

In order to evaluate the performance of proposed hybrid schemes, utilizing LSs and KLs in trust-aware ELRSs, we are conducting two experiments in the proposed collaborative filtering framework are as follows: 1. Evaluation of system’s performance based on proposed scheme using MAE and Coverage measures 2. Effect of number of varying neighborhood size on system’s performance

2- Experimental setup

Since there is no well known dataset publically accessible for research (Bobadilla et al., 2009) in the domain of ELRSs, we mapped well known datasets MovieLens and Jester from the RSs domain to ELRSs domain (here after referred as EL dataset) for our experimentation. 

3- MovieLens dataset consists of 

100,000 ratings provided by 943 learners on 1682 movies. The rating scale follows the 1-bad, 2-average, 3-good, 4-very good and 5-excellent. Each user has rated at least 20 movies in this dataset. Jester dataset provides 4.1 million ratings by 73421 users on 100 jokes. The ratings are continuous and lie in range -10 to 10. 

4- MovieLens (ML) and Jester are different from 

Each other, since the sparsity level of MovieLens is quite high whereas the jester dataset is dense. We mapped movies of ML dataset and jokes of jester dataset as teachers, books and subjects and the ratings to movies and jokes are considered as ratings provided by learners to teachers, books and subjects. 

Because of the requirement of learning styles and test scores in our proposed work, we assigned learning styles as well as test scores for each learner randomly. LSs are generated randomly using ILS questionnaire. For establishing correlation between LSs of learners and their ratings patterns, we generate 

5- LSs patterns for similar learners on

 The basis of the questionnaire and select pattern with high correlation. To compare proposed schemes more thoroughly, we conducted the experiments under several configurations. We randomly selected subsets of 150-550 with increment of 100 learners called EL150, EL250, EL350, EL450 and EL550 from both of the datasets respectively. 

This is to illustrate the effectiveness of the proposed scheme under varying number of participating learners. Each of these sub datasets was randomly split into 60% training and 40% test data (Anand & Bharadwaj, 2011). 

6- The ratings of the items in the test 

Set are treated as items unseen by the active learner while the ratings in the training set are used for neighborhood construction and for prediction of ratings. For each dataset, experiments were run 20 times to eliminate the effect of any bias in the data.

 7- Performance evaluation 

In order to test the performance of our schemes, we measure system’s accuracy using two evaluation metrics, namely the mean absolute error (MAE) and the total coverage of system. The MAE measures the deviation of predictions generated by the RS from the true ratings specified by the learner. 

On the other hand, coverage measures the percentage of resource for which a RS can provide predictions The MAE for the active learner 𝑙𝑎 (Breese et al., 1998) is as follows:

 𝑀𝐴𝐸(𝑙𝑎 ) = 1 |𝐿𝐿𝑖| ∑ �𝑝𝑝𝑝𝑝𝑖𝑖,𝑘 − 𝑝𝑝𝑖𝑖,𝑘� |𝐿𝐿𝑖| 𝑘=1 , 

(8) where i is the resource and 𝑆𝑖𝑖 is the cardinality of the test ratings set of learner 𝑙𝑎 ratings. The total MAE over all the active learners 𝑁𝐴 can be computed as: 

 𝑀𝐴𝐸 = 1 𝑁𝐴 ∑ 𝑀𝐴𝐸(𝑙𝑎) 𝑁𝐴 𝑎=1 (9) 

Coverage: It is measured as number of resources for which RS can generate prediction over the total number of unseen resources. 𝐶𝑜𝑣𝑒𝑝𝑝𝑎𝑔𝑒 = ∑ 𝑝𝑖 𝑁𝐴 𝑖 ∑ 𝐿𝐿𝑖 𝑁𝐴 𝑖  

here, 𝑝𝑝𝑖𝑖 is the total number of predicted resources for active learner 𝑙𝑎. Low coverage value indicates that RS will not able to suggest the learner with many of resource he has not rated.

Experiment 1 

In order to reveal the performance of system based on proposed schemes, we conduct experiments using datasets EL150, EL250, EL350, EL450 and EL550 mapped from MovieLens and Jester. By fixing the neighborhood size, we compare MAE and coverage. Using an appropriate value of α, we measured the effect of CF-PR, CF-KL, CF-LS, CF-LS-KL and TRCF-LS-KL. 

8- Analysis of the results 

Tables 4 and 5 show that TRCF-LS-KL outperforms CF-LS-KL, CF-LS, CF-KL and CF-PR for all the different configurations generated from MovieLens and Jester datasets respectively. The total MAE for TRCF-LS-KL is always smaller than CF-LS-KL, CF-LS, CF-KL and CF-PR schemes. 

Tables 6 and 7 show that TRCF-LS-KL outperforms CF-LS-KL, CF-LS, CF-KL and CF-PR generated from MovieLens and Jester datasets respectively. The coverage for TRCF-LS-KL is always greater than CF-LS-KL, CF-LS, CF-KL and CF-PR schemes.

Experiment 2 

In order to evaluate the effect of neighborhood size on performance of CF-PR, CF-KL, CF-LS, CF-LS-KL and TRCF-LS-KL, we conducted an experiment where we varied the number of nearest neighbors that were used and computed MAE for each scheme. 

We showed only the results for configuration EL250 and EL450 mapped from MovieLens and Jester datasets respectively. Results are shown in Figure 8(a) and Figure 8(b). We can observe that the size of neighborhood does affect the performance. All schemes improved the accuracy of prediction as neighborhoods size increases from 5 to 30 with increment 5.

Analysis of the results 

Figure 8 (a) and Figure 8 (b) show that CF-KL performs better than CF-LS when the size of neighborhood lies in between 5 to 10. TRCF-LS-KL scheme is always better than other schemes. When the neighborhood size increases, MAE of all schemes decreases. 

Discussion 

The experimental results have clearly demonstrated that our proposed schemes namely, CF-KL, CF-LS, CF-LS-KL and TRCF-LS-KL have produced high predictive accuracy and better coverage on two different datasets (MovieLens and Jester) as compared to traditional collaborative filtering CF-PR. For TRCF-LS-KL, the mean MAE of all sample datasets of 

MovieLens decreases approximately by 8.5 % and the mean coverage increases by 18%. The improvement in accuracy via MAE and coverage is seen significantly for Jester dataset also. This improvement is possible because of the adaptation of trustworthy as well as similar learners in terms of their 

LSs and KLs in our proposed schemes. The proposed ELRS (TRCF-LS-KL) would be quite effective for providing resource recommendations to learners registered in online courses through e-learning systems.

 Conclusion 

E-learning recommender systems (ELRSs) need to consider specific demands and preferences of learners. Our proposed ELRS clearly demonstrates that trustworthy as well as similar learners generate effective resource

recommendations considering their learning styles (LSs) and knowledge levels (KLs). Experimental results reveal that the incorporation of trust in ELRS based on LSs and KLs has significantly outperformed other traditional schemes in terms of accuracy and coverage. 

Even though the results reveal that the proposed scheme, TRCF-LS-KL has produced better accuracy and coverage, further studies are planned to extend the TRCF-LS-KL in the following two different directions: Firstly, LS vector is fixed in our proposed system during learning process. 

It would be more promising by adapting the dynamic behavior of LSs and incorporating emotional states of learners such as sad, anger etc for further enhancing recommendation quality of our proposed system. Further, our proposed system is error prone to the cold start and the sparsity problems. 

In future, we are planning to develop a hybrid ELRS for alleviating these problems by combining our system with content based filtering and incorporating trust propagation schemes into our system. The results presented in this study are still preliminary due to the lack of publically available datasets; it would be interesting to establish the effectiveness of our proposed system on future datasets with different data characteristics.