People

We are also so pleased to welcome Daan as a PhD student working on the SNSF D3: Digital Deliberative Democracy project.

Research

Our research was published across a range of conferences and journals:

• Athina, Oana, Prof. Bernstein, and externals published the work Algorithmically Mediated User Relations: Exploring Data’s Relationality in Recommender Systems (Regulatable ML Workshop, NeurIPS ‘23)
• Florian, Cristina, and Prof. Bernstein published the paper DREAM: Deployment of Recombination and Ensembles in Argument Mining (EMNLP ‘23)
• Kathrin and Oana collaborated with externals on the project Interaction Tasks for Explainable Recommender Systems (Eurovis ‘23)
• Luca together with externals published the papers:
  • Comparing Interactive Retrieval Approaches at the Lifelog Search Challenge 2021 (IEEE Access)
  • Exploring Effective Interactive Text-Based Video Search in vitrivr (MMM ‘23 VBS)
  • Traceable Asynchronous Workflows in Video Retrieval with vitrivr-VR (MMM ‘23 VBS)
  • A Comparison of Video Browsing Performance between Desktop and Virtual Reality Interfaces (ICMR ‘23)
  • The Best of Both Worlds: Lifelog Retrieval with a Desktop-Virtual Reality Hybrid System (ICMR ‘23 LSC)
  • Gesture retrieval and its application to the study of multimodal communication (International Journal on Digital Libraries)
  • Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th VBS (Multimedia Systems Journal)
  • Novice-Friendly Text-based Video Search with vitrivr (CBMI’23)
  • Spatially Localised Immersive Contemporary and Historic Photo Presentation on Mobile Devices in Augmented Reality (SUMAC’23)
• Luca, along with Oana, Svenja, Florian, Ruijie, and Prof. Bernstein participated in the Lifelog Challenge with the work Multi-Mode Clustering for Graph-Based Lifelog Retrieval (ICMR ‘23 LSC)
• Lucien, Prof. Bernstein, and externals published the short paper Deliberative Diversity for News Recommendations: Operationalization and Experimental User Study (RecSys ‘23)
• Oana together with externals published the papers:
  • Collect, measure, repeat: Reliability factors for responsible AI data collection (best paper honorable mention, HCOMP ‘23)
  • Evaluating explainable social choice-based aggregation strategies for group recommendation (UMAP ‘23)
  • Viewpoint diversity in search results (ECIR ‘23)
  • Dataperf: Benchmarks for data-centric ai development (NeurIPS ‘23, Datasets and Benchmarks track)
• Rosni with externals published the work Med-EASi: Finely Annotated Dataset and Models for Controllable Simplification of Medical Texts (AAAI ‘23)
• Ruijie, together with UZH student Zhiruo Zhang, Luca, Florian, Prof. Bernstein and collaborated on the work NLQxform: A Language Model-based Question to SPARQL Transformer (Scholarly QALD@ISWC 2023)
• DDIS alumni Suzanne, together with UZH student Vasiliki Arpatzoglou, Luca, and Prof. Bernstein published the work Trolleys, crashes, and perception – a survey on how current autonomous vehicles debates invoke problematic expectations (AI and Ethics)
• UZH student Viktor Lakic, together with Luca, and Prof. Bernstein published the work Link-Rot in Web-Sourced Multimedia Datasets (best special session paper award MMM ‘23)

Grants & Scholarships

Rosni was awarded the AAAI 2023 Student Scholarship and Volunteer Program and Lucien the GRC Travel Grant! Also, Athina and Fynn were selected for the DSI Excellence Program!

Events

During 2023, we organized and participated in numerous events! Specifically:

• Cristina presented in the DigDem Workshop ‘23 
• Lucien and Prof. Bernstein participated in The Impact of Algorithm-Powered News Recommendation Systems (ICA ‘23) and co-organized NORMalize: The First Workshop on Normative Design and Evaluation of Recommender Systems (RecSys ’23)
• Luca co-organized the Video Browser Showdown (VBS) ‘23 and the Lifelog Search Challenge (LSC) ‘23
• Oana co-organized QUARE: 2nd Workshop on Measuring the Quality of Explanations in Recommender Systems (RecSys ‘23), 5th Workshop on Explainable User Models and Personalised Systems (UMAP’23), The ART of Safety: Workshop on Adversarial testing and Red-Teaming for generative AI (AACL’23), and the Adversarial Nibbler challenge

Student Theses and Projects

The UZH student Konstantina Timoleon helped us to further improve the script of the Advanced Topics in AI lecture.

Additionally, the following master students completed their theses in DDIS:

• Zhiruo Zhang – Answer Aggregation and Verbalization for Complex Question Answering
• Domenic Fürer – Multimodal Feature Spaces
• Jan Willi – Unified Multimedia Segmentation
• Nina Willis – Human Perception and Memory in Interactive Video Retrieval

Finally, in 2023 3 BSc theses and 5 Master Projects were completed under DDIS supervision!

May 2024 be prosperous!

The paper’s problem of interest is the identification of an epidemic’s patient zero given a network of contacts and a set of infected individuals, under the assumptions that the infection states of only a few individuals are initially observed and the epidemic has evolved. To tackle this issue, they formulate the problem as an active querying problem and propose a number of active querying strategies inspired by active learning. Their results suggest that in the limited information scenario it is possible to achieve source inference performance comparable to when the infection states of all individuals are observed.

A paper based on controllable models for simplifying medical text, co-authored by our colleague Rosni Kottekulam Vasu and external collaborators, was accepted at the 37th AAAI Conference on Artificial Intelligence! The paper is titled “Med-EASi: Finely Annotated Dataset and Models for Controllable Simplification of Medical Texts” and was jointly conducted with Chandrayee Basu, Michihiro Yasunaga from Stanford University, and Qian Yang from Cornell University.

The paper’s vision is an interactive automatic medical text simplification system, which can enable medical practitioners and patients to simplify the contents of a text or conversation selectively and have controllability over the type of desired textual transformations.

Rosni presented the work at AAAI, orally and on a poster and was accepted to the 2023 AAAI student scholarship and volunteer program. Congratulations!

In his Bachelor Thesis, Viktor Lakic investigated the decay happening in datasets when the resources that Web-URLs point to become unavailable. This Link-Rot can cause problems for reproducibility, as datasets can shrink over time, potentially changing the outcome of experiments which use them. A paper based on the data that Viktor collected in his thesis, co-authored by Luca Rossetto and Abraham Bernstein, was recently presented at the 2023 International Conference on Multimedia Modeling in the Special Session on ‘Multimedia Datasets for Repeatable Experimentation’. The paper was awarded the ‘Best Special Session Paper Award’, honoring the best contribution across all special sessions of the conference. Congratulations!

Staff Updates

We were sad to see our colleague Romana Pernisch leave DDIS, but it was for a very good reason: Romana Pernisch successfully defended her PhD thesis “Mind the Change, Bridge the Gap Investigating the Impact of Ontology Evolution on Materialisations and Embeddings” supervised by Prof. Bernstein. Romana is currently a postdoc at Vrije Universiteit (VU) Amsterdam. We wish her all the best!

We are very happy that Kathrin Wardatzky decided to join us in 2021 to pursue a PhD on explainable AI.

Research

Our research got published in a variety of venues resulting in a total of 20 publications:

• Suzanne Tolmeijer, Lucien Heitz and Prof. Bernstein, together with external colleagues, published the Dagstuhl Manifesto “Diversity in News Recommendations“.
• Suzanne Tolmeijer, Prof. Bernstein, and external colleagues published the conference paper “Second Chance for a First Impression? Trust Development in Intelligent System Interaction” at User Modeling, Adaptation and Personalization (UMAP).
• Suzanne Tolmeijer, Prof. Bernstein, and external colleagues published the extended abstract “Female by Default?–Exploring the Effect of Voice Assistant Gender and Pitch on Trait and Trust Attribution” at Human Factors in Computing Systems (CHI).
• Matthias Baumgartner, Daniele Dell’Aglio and Prof. Bernstein published the workshop paper “Entity Prediction in Knowledge Graphs with Joint Embeddings” at the Workshop Graph-Based Methods for Natural Language Processing (TextGraphs).
• External collaborators and Rosni Vasu published the workshop paper “Automatic Medical Text Simplification: Challenges of Data Quality and Curation” at the Human Partnership with Medical Artificial Intelligence Workshop within the AAAI Fall Symposium.
• External collaborators and Cristina Sarasua published the journal article “The Impact of Task Abandonment in Crowdsourcing” at IEEE Transactions on Knowledge and Data Engineering.
• Martin Sterchi, Cristina Sarasua, Prof. Bernstein and other external colleagues published the journal article “Outbreak Detection for Temporal Contact Data” at the Applied Network Science Journal.
• Narges Ashena, Daniele Daniele Dell’Aglio, and Prof. Bernstein published the conference paper “Understanding ε for Differential Privacy in Differencing Attack Scenarios” at EAI-SecureComm 2021.
• Narges Ashena, Matthias Baumgartner, Lucien Heitz, Romana Pernisch, Florian Ruosch, Luca Rossetto, and Prof. Bernstein published the benchmark paper “VideoGraph – Towards Using Knowledge Graphs for Interactive Video Retrieval” at the Video Browser Showdown.
• Matthias Baumgartner, Lucien Heitz, Luca Rossetto, Ruijie Wang, Prof. Bernstein, and external collaborators published the workshop paper “Exploring Graph-querying approaches in LifeGraph” at the Lifelog Search Challenge. At the same venue, Luca Rossetto published other two workshop papers with external collaborators: “Exploring Intuitive Lifelog Retrieval and Interaction Modes in Virtual Reality with vitrivr-VR” and “Interactive Multimodal Lifelog Retrieval with vitrivr at LSC 2021“.
• Luca Rossetto and external collaborators published the journal article “On the User-centric Comparative Remote Evaluation of Interactive Video Search Systems” at IEEE Multimedia. 
• Luca Rossetto, Prof. Bernstein and external collaborators also published the demo paper “A system for interactive multimedia retrieval evaluations” and the conference paper “Considering Human Perception and Memory in Interactive Multimedia Retrieval Evaluations” at the International Conference on Multimedia Modelling, and a technical report on Arxiv “Insights on the V3C2 Dataset“.
• Moreover, some external collaborators and Luca Rossetto published the (i) benchmark paper “Competitive interactive video retrieval in virtual reality with vitrivr-VR” at the Video Browser Showdown, (ii) the journal article “Is the Reign of Interactive Search Eternal? – Findings from the Video Browser Showdown 2020” at ACM Transactions on Multimedia Computing, Communications, and Applications, (iii) the conference paper “Gesture of Interest: Gesture Search for Multi-Person, Multi-Perspective TV Footage» at International Conference on Content-Based Multimedia Indexing, and (iv) the conference paper “Introduction to the Fourth Annual Lifelog Search Challenge, LSC’21” at the International Conference on Multimedia Retrieval.

Grants

2021 was extremely successful in terms of funding acquisition:

• Luca Rossetto received an SNSF Ambizione grant for the MediaGraph project.
• Prof. Bernstein and collaborators received an SNSF Sinergia grant for the project Large-Scale Political Participation: Issue Identification, Deliberation, and Co-creation —a collaboration with Cristina Sarasua, UZH Prof. Marco Steenbergen and UZH Prof. Felix Uhlmann, as well as Prof. Gianluca Demartini from the University of Queensland (UQ).
• Lucien Heitz and Prof. Bernstein received a BAKOM grant for a second round of the news experiment.
• Rosni Vasu and her collaborators from Stanford and Cornell University received a Toloka research grant for crowdsourcing experiments.

Events

Networking and disseminating our work at scientific events is one of the most exciting activities for us, scientists. While in 2021 we continued to attend events mostly online due to the COVID-19 situation, we had the chance to participate in very interesting workshops and conferences:

Suzanne Tolmeijer, Lucien Heitz and Prof. Bernstein participated in the workshop “AI, Jounalism & Democracy: Exploring the Normative Dimension” and the conference “Future of Journalism“, presenting their work on news recommendations. Lucien Heitz also participated in the “OSCE AI & FoE Workshops on Media Pluralism“. Luca Rossetto co-organized the “2nd International Workshop on Video Retrieval Methods and Their Limits” and the “Lifelog Search Challenge“. Matthias Baumgartner, Lucien Heitz, Luca Rossetto, Ruijie Wang and Prof. Bernstein also contributed with a system to the “Lifelog Search Challenge“. Narges Ashena, Matthias Baumgartner, Lucien Heitz, Romana Pernisch, Florian Ruosch, Luca Rossetto, and Prof. Bernstein contributed a benchmark to the “Video Browser Showdown“. Suzanne Tolmeijer participated in CHI 2021 and UMAP 2021 presenting her research, and she had the honour of joining the Heidelberg Laureate Forum where she had the opportunity to interact with recipients of the Turing Award.

Cristina Sarasua, Kathrin Wardatzky, and Rosni Vasu participated in CSCW 2021, Ruijie Wang participated in ISWC 2021, and Rosni Vasu also participated in the AAAI Fall Symposium 2021, and the HCOMP Graduate Consortium 2021. Narges Ashena presented her research at EAI SecureComm 2021.

Student Theses and Projects

We supervised three master projects, four bachelor theses, and 11 master theses.

In the area of news recommender systems: Valérie Nyffeler, Julian Croci, Madhav Sachdeva, Lukas Yu successfully completed their master project “Informfully Full Stack News App Extension” and Emanuel Graf, Vladimir Donkov completed their master project “Multimedia Extensions for Informfully” both supervised by Lucien Heitz. Nick Kipfer completed the bachelor thesis “Automatic Selection of Illustrative Pictures for News Articles” co-supervised by Lucien Heitz and Luca Rossetto. Marco Heiniger completed the bachelor thesis “Recommender System for Portfolio Management” supervised by Lucien Heitz. Lukas Yu and Lukas Grässle are working on their master theses “Style Transfer Algorithm for Online News” and “Recommender Systems” supervised by Lucien Heitz and plan to finish them in 2022.

In the area of multimedia understanding: Patrick Düggelin completed the master thesis “Voice isolation, speech transcription and speaker re-identification in video”, Alexander Theus completed the bachelor thesis “Scene Text Extraction for Retrieval of Visual Multimedia”, Amos Neculau completed the master thesis “Multi-Domain Media Segmentation”” Lawand Muhamad completed the master thesis “Approximate Boolean Retrieval”, Simon Widmer completed the master thesis “Large-scale Active Learning for Concept Detection in Video” —all supervised by Luca Rossetto. A paper based on the thesis by Alexander Theus has been recently accepted at MMM’22. Lutharsanen Kunam is planning to finish the master thesis “High Level Semantic Video Understanding” with Luca Rossetto in 2022.

In the context of the CrowdAlytics project: Joel Watter completed his bachelor thesis “The Argument Annotator Pipeline – Generate Visually Annotated Documents” supervised by Florian Ruosch. Ning Xie and Rinor Sefa are working on the “CrowdAlytics Annotation Framework” as a master project supervised by Cristina Sarasua, Florian Ruosch, Rosni Vasu, and Dhivyabharathi Ramasamy.

Moreover, Terézia Bucková completed her master thesis “Supervised and Unsupervised Alignment of Knowledge Graphs with pre-trained embeddings” supervised by Matthias Baumgartner and Daniele Dell’Aglio, and Badrie L. Persaud completed the master thesis “Human Perception of Privacy: Visualizing Epsilon for Differential Privacy” supervised by Narges Ashena. Vasiliki Arpatzoglou is working on the master thesis “Autonomous Car Acceptance” supervised by Suzanne Tolmeijer and Luca Rossetto, and Fan Feng is currently working on the master thesis “Natural Language Question Answering via Knowledge Graph Reasoning” supervised by Ruijie Wang.

Announcements

We have several open positions. If you are interested in pursuing a PhD and working with us, please have a look at the current job ads: https://www.ifi.uzh.ch/en/ddis/jobs.html

We look forward to the new colleagues, projects, experiments, and papers to come!