The paper presents the results of a user study investigating the efficacy of various interactive data visualizations in explaining the functioning of Differential Privacy and the adjustment of its key parameter, ε. The study surveyed 426 participants, revealing valuable insights into communicating complex and technical privacy-preserving techniques to users. It was presented by Narges in San Francisco in May 2024.

People

We are also so pleased to welcome Daan as a PhD student working on the SNSF D3: Digital Deliberative Democracy project.

Research

Our research was published across a range of conferences and journals:

• Athina, Oana, Prof. Bernstein, and externals published the work Algorithmically Mediated User Relations: Exploring Data’s Relationality in Recommender Systems (Regulatable ML Workshop, NeurIPS ‘23)
• Florian, Cristina, and Prof. Bernstein published the paper DREAM: Deployment of Recombination and Ensembles in Argument Mining (EMNLP ‘23)
• Kathrin and Oana collaborated with externals on the project Interaction Tasks for Explainable Recommender Systems (Eurovis ‘23)
• Luca together with externals published the papers:
  • Comparing Interactive Retrieval Approaches at the Lifelog Search Challenge 2021 (IEEE Access)
  • Exploring Effective Interactive Text-Based Video Search in vitrivr (MMM ‘23 VBS)
  • Traceable Asynchronous Workflows in Video Retrieval with vitrivr-VR (MMM ‘23 VBS)
  • A Comparison of Video Browsing Performance between Desktop and Virtual Reality Interfaces (ICMR ‘23)
  • The Best of Both Worlds: Lifelog Retrieval with a Desktop-Virtual Reality Hybrid System (ICMR ‘23 LSC)
  • Gesture retrieval and its application to the study of multimodal communication (International Journal on Digital Libraries)
  • Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th VBS (Multimedia Systems Journal)
  • Novice-Friendly Text-based Video Search with vitrivr (CBMI’23)
  • Spatially Localised Immersive Contemporary and Historic Photo Presentation on Mobile Devices in Augmented Reality (SUMAC’23)
• Luca, along with Oana, Svenja, Florian, Ruijie, and Prof. Bernstein participated in the Lifelog Challenge with the work Multi-Mode Clustering for Graph-Based Lifelog Retrieval (ICMR ‘23 LSC)
• Lucien, Prof. Bernstein, and externals published the short paper Deliberative Diversity for News Recommendations: Operationalization and Experimental User Study (RecSys ‘23)
• Oana together with externals published the papers:
  • Collect, measure, repeat: Reliability factors for responsible AI data collection (best paper honorable mention, HCOMP ‘23)
  • Evaluating explainable social choice-based aggregation strategies for group recommendation (UMAP ‘23)
  • Viewpoint diversity in search results (ECIR ‘23)
  • Dataperf: Benchmarks for data-centric ai development (NeurIPS ‘23, Datasets and Benchmarks track)
• Rosni with externals published the work Med-EASi: Finely Annotated Dataset and Models for Controllable Simplification of Medical Texts (AAAI ‘23)
• Ruijie, together with UZH student Zhiruo Zhang, Luca, Florian, Prof. Bernstein and collaborated on the work NLQxform: A Language Model-based Question to SPARQL Transformer (Scholarly QALD@ISWC 2023)
• DDIS alumni Suzanne, together with UZH student Vasiliki Arpatzoglou, Luca, and Prof. Bernstein published the work Trolleys, crashes, and perception – a survey on how current autonomous vehicles debates invoke problematic expectations (AI and Ethics)
• UZH student Viktor Lakic, together with Luca, and Prof. Bernstein published the work Link-Rot in Web-Sourced Multimedia Datasets (best special session paper award MMM ‘23)

Grants & Scholarships

Rosni was awarded the AAAI 2023 Student Scholarship and Volunteer Program and Lucien the GRC Travel Grant! Also, Athina and Fynn were selected for the DSI Excellence Program!

Events

During 2023, we organized and participated in numerous events! Specifically:

• Cristina presented in the DigDem Workshop ‘23 
• Lucien and Prof. Bernstein participated in The Impact of Algorithm-Powered News Recommendation Systems (ICA ‘23) and co-organized NORMalize: The First Workshop on Normative Design and Evaluation of Recommender Systems (RecSys ’23)
• Luca co-organized the Video Browser Showdown (VBS) ‘23 and the Lifelog Search Challenge (LSC) ‘23
• Oana co-organized QUARE: 2nd Workshop on Measuring the Quality of Explanations in Recommender Systems (RecSys ‘23), 5th Workshop on Explainable User Models and Personalised Systems (UMAP’23), The ART of Safety: Workshop on Adversarial testing and Red-Teaming for generative AI (AACL’23), and the Adversarial Nibbler challenge

Student Theses and Projects

The UZH student Konstantina Timoleon helped us to further improve the script of the Advanced Topics in AI lecture.

Additionally, the following master students completed their theses in DDIS:

• Zhiruo Zhang – Answer Aggregation and Verbalization for Complex Question Answering
• Domenic Fürer – Multimodal Feature Spaces
• Jan Willi – Unified Multimedia Segmentation
• Nina Willis – Human Perception and Memory in Interactive Video Retrieval

Finally, in 2023 3 BSc theses and 5 Master Projects were completed under DDIS supervision!

May 2024 be prosperous!

Ruijie implemented NLQxform, a question-answering system to answer natural language questions on scholarly knowledge graphs. The system integrates a fine-tuned transformed-based BART model to translate natural language questions to SPARQL queries. Check out the NLQxform poster for more information!

The paper’s problem of interest is the identification of an epidemic’s patient zero given a network of contacts and a set of infected individuals, under the assumptions that the infection states of only a few individuals are initially observed and the epidemic has evolved. To tackle this issue, they formulate the problem as an active querying problem and propose a number of active querying strategies inspired by active learning. Their results suggest that in the limited information scenario it is possible to achieve source inference performance comparable to when the infection states of all individuals are observed.

People

In 2022, three members of the DDIS group successfully defended their PhD Thesis!

In February, Suzanne defended her thesis “The Right Thing To Do? Artificial Intelligence for Ethical Decision Making”. She has since joined the University of Hamburg as a postdoctoral researcher in the group of Prof. Bittner.

Matthias successfully defended his thesis “How to Compare Apples to Oranges: Integrating Heterogeneous Data Sources with Representation Learning” in August.

Finally, in November, Martin defended his thesis “Epidemic Spreading on Networks” and he is currently a lecturer for Business Analytics at the School of Business FHNW.

We wish them all the best in their future endeavours! 

In the past few months, we were also pleased to welcome Oana as a postdoctoral researcher, as well as Svenja, Athina, Fynn, and Mikla as PhD students!

Research & Projects

Our research, ranging from multimedia retrieval and crowdsourcing to digital democracy and responsible AI, got published in a variety of conferences and journals:

• Dhivya, Cristina, Prof. Bernstein, and Prof. Bacchelli published the paper Workflow Analysis of Data Science Code in Public GitHub Repositories in the Empirical Software Engineering Journal.
• Florian, Cristina, and Prof. Bernstein published the paper BAM: Benchmarking Argument Mining on Scientific Documents in the workshop proceedings in the AAAI Conference on Artificial Intelligence.
• Luca and external collaborators published the paper Interactive video retrieval evaluation at a distance: comparing sixteen interactive video search systems in a remote setting at the 10th Video Browser Showdown in the International Journal of Multimedia Information Retrieval, as well as the papers A Task Category Space for User-Centric Comparative Multimedia Search Evaluations, Multi-Modal Interactive Video Retrieval with Temporal Queries, and Multi-modal Video Retrieval in Virtual Reality with vitrivr-VR in the International Conference on Multimedia Modeling. Luca contributed also to the workshop paper vitrivr at the Lifelog Search Challenge 2022 at ACM ICMR. Moreover, together with Prof. Bernstein and external collaborators Luca published the paper An Asynchronous Scheme for the Distributed Evaluation of Interactive Multimedia Retrieval in the Workshop on Interactive Multimedia Retrieval at ACM Multimedia.
• Lucien, Suzanne, Prof. Bernstein, and external collaborators published the paper Benefits of Diverse News Recommendations for Democracy: A User Study in the Digital Journalism Journal.
• Lucien contributed to the Spotlight on Artificial Intelligence and Freedom of Expression: A Policy Manual, an initiative of the Office of the Representative on Freedom of the Media Organization for Security and Co-operation in Europe.
• Matthias, Daniele, Prof. Bernstein, and external collaborators published the paper Towards the Web of Embeddings: Integrating multiple knowledge graph embedding spaces with FedCoder in the Journal of Web Semantics.
• Oana and external collaborators published the papers Comprehensive viewpoint representations for a deeper understanding of user interactions with debated topics in the ACM SIGIR Conference on Human Information Interaction and Retrieval, Understanding the Role of Explanation Modality in AI-assisted Decision-making in the ACM Conference on User Modeling, Adaptation and Personalization, and Humans disagree with the IoU for measuring object detector localization error in the IEEE International Conference on Image Processing.
• The student Alexander Theus together with Luca and Prof. Bernstein published the paper HyText-a Scene-Text Extraction Method for Video Retrieval in the International Conference on Multimedia Modeling.
• The student Lutharsanen Kunam together with Luca and Prof. Bernstein published their work A Multi-Stream Approach for Video Understanding in the Deep Video Understanding Challenge at ACM Multimedia.

Moreover, the MediaGraph SNSF Ambizione and the Digital Deliberative Democracy (D^3) SNSF Sinergia projects have successfully started!

Grants

• Cristina and Lucien received a SNSF Scientific Exchange Grant.
• Cristina, Oana, Luca, and Prof. Bernstein received a UZH Teaching Innovation Fund to extend the Speakeasy platform.
• Cristina and Lucien, together with external colleagues, received a GRC Short Grant.
• Lucien completed the DSI Excellence Program.

Events

Throughout 2022, we had the opportunity to organise and participate in numerous events, in particular:

• Florian participated in the Dagstuhl workshop Towards a Unified Model of Scholarly Argumentation.
• Luca co-organised the 5th Lifelog Search Challenge Workshop at ACM ICMR, the Workshop on Interactive Multimedia Retrieval at ACM Multimedia, and the tutorial “Open Challenges of Interactive Video Search and Evaluation” at ACM Multimedia.
• Lucien participated in ICA 2022.
• Oana co-organized the Workshop on Explainable User Modeling and Personalized Systems as well as the Workshop on Measuring the Quality of Explanations in Recommender Systems.
• Athina, Kathrin, Lucien, and Oana participated in the Lorentz Center workshop Diverse News Recommenders: from concept to implementation organised by Prof. Bernstein and external colleagues.
• Cristina, Fynn, Mikla, and Prof. Bernstein co-organised the Dagstuhl workshop Challenges and Opportunities of Democracy in the Digital Society. Also, Cristina, Fynn, and Mikla participated in the Digital Democracy Workshop and together with Lucien organised the Liquid Democracy Workshop.

Student Theses and Projects

The UZH student Jan Will is working with Cristina and Rosni on the CrowdAlytics Annotation Framework v2.0. Additionally, in collaboration with Cristina and Luca, Jan is extending the Speakeasy platform.

Moreover the UZH student Laurin van den Bergh is working with Cristina and Luca on the script of the Advanced Topics in AI lecture.

Additionally, the following master students completed their theses in DDIS:

• Vasiliki Arpatzoglou – Autonomous Car Acceptance
• Fan Feng – Natural Language Question Answering via Knowledge Graph Reasoning
• Emanuel Graf – Conversational Crowdsourcing for hypothesis generation in data science
• Lutharsanen Kunam – High Level Semantic Video Understanding
• Lukas Yu – Style Transfer Algorithm for Online News

Finally, in 2022 we supervised 4 BSc theses and 2 Master Projects!

Happy New Year 2023!

For five days, scholars in computer science, political science, law, and communication sciences discussed current challenges and future opportunities of online participation, political communication, and online deliberation.

We wish Matthias all the best in his future endeavors!

“Evaluation of Algorithms for Interaction-Sparse Recommendations: Neural Networks don’t Always Win“

Authors: Yasamin Klingler, Claude Lehmann, João Pedro Monteiro, Carlo Saladin, Abraham Bernstein, Kurt Stockinger

Abstract: “In recent years, top-K recommender systems with implicit feedback data gained interest in many real world business scenarios. In particular, neural networks have shown promising results on these tasks. However, while traditional recommender systems are built on datasets with frequent user interactions, insurance recommenders often have access to a very limited amount of user interactions, as people only buy a few insurance products. In this paper, we shed new light on the problem of top-K recommendations for interaction-sparse recommender problems.
In particular, we analyze six different recommender algorithms, namely a popularity-based baseline and compare it against two matrix factorization methods (SVD++, ALS), one neural network approach (JCA) and two combinations of neural network and factorization machine approaches (DeepFM, NeuFM). We evaluate these algorithms on six different interaction-sparse datasets and one dataset with a less sparse interaction pattern to elucidate the unique behavior of interaction-sparse datasets.
In our experimental evaluation based on real-world insurance data, we demonstrate that DeepFM shows the best performance followed by JCA and SVD++, which indicates that neural network approaches are the dominant technologies. However, for the remaining five datasets we observe a different pattern. Overall, the matrix factorization method SVD++ is the winner. Surprisingly, the simple popularity-based approach comes out second followed by the neural network approach JCA. In summary, our experimental evaluation for interaction-sparse datasets demonstrates that in general matrix factorization methods outperform neural network approaches. As a consequence, traditional well-established methods should be part of the portfolio of algorithms to solve real-world interaction-sparse recommender problems.”

The paper can be read here.

“HyText – A Scene-Text Extraction Method for Video Retrieval.“

Authors: Alexander Theus, Luca Rossetto, and Abraham Bernstein

Abstract: “Scene-text has been shown to be an effective query target for video retrieval applications in a known-item search context. While much progress has been made in scene-text extraction from individual pictures, the special case of video has so far received less attention. This paper introduces HyText, a scene-text extraction method for video with a focus on retrieval applications. HyText uses intermittent scene-text detection in combination with bi-directional tracking in order to increase throughput without reducing detection accuracy.”

You can read the full paper here.