The CrowdRec project is closely connected to CLEF NEWSREEL, a living lab for benchmarking news recommendations in real-time. CLEF, short for Conference and Labs of Evaluation Form, is one of the main academic campaigns for benchmarking information access systems. Each year, various benchmarking scenarios, referred to as labs, are offered in the context of CLEF.
The NEWSREEL (News Recommendation Evaluation Lab) lab is a benchmarking task that focuses on benchmarking news recommendations in a live data stream of news articles. The lab started a few weeks before the beginning of CrowdRec as an initiative from CrowdRec members of TU Berlin. By participating in NEWSREEL, researchers could evaluate news recommendation techniques over a period of a few months, i.e., the system has been running for the whole duration of CrowdRec. Results were presented in September 2014 at the annual CLEF workshop in Sheffield.
See here for the Overview of CLEF NEWSREEL 2014