Computer Science Department

Bucknell University

Lewisburg, PA 17837

May 2014

Please follow this link to see the submission instructions.

This set of assignment is designed for you to get familiar with various evaluation mechanisms in IR.

- Given the set of data in the table,

- Compute the recall/precision rate.
- Graph the result for the recall/precision rate.
- Compute the corresponding F-measure for each of the recall points (0.0 to 1.0). That is, the parameters α = 0.5 and β = 1.
The total relevant documents in a document set is 10. The top 10 retrieved documents are listed with the ones that are relevant marked.

Order Doc ID Relevant 1 586 X 2 357 3 358 X 4 108 X 5 345 6 114 7 555 X 8 888 9 167 10 666 X Compute the Kappa statistics for the following data.

Judge 2 Relevance Yes No Total Judge 1 Yes 200 15 215 Relevance No 20 50 70 Total 220 65 285