https://whatsup.linguistik.uzh.ch/ 2026-04-14T22:01:08+00:00 https://whatsup.linguistik.uzh.ch/ https://whatsup.linguistik.uzh.ch/_media/wiki/logo.png text/html 2022-06-27T07:21:45+00:00 Anonymous (anonymous@undisclosed.example.com) https://whatsup.linguistik.uzh.ch/01_corpus/02_preprocessing/01_anonymization?rev=1656314505&do=diff 1.2.1 Anonymization General privacy People who write WhatsApp chats can be recognized either by the stories they tell or by the names and places they mention. While we had no way to address the former, we addressed the latter by means of computational linguistics. If you happen to recognize informants based on the remaining information, we ask you to comply with common text/html 2022-06-27T07:21:45+00:00 Anonymous (anonymous@undisclosed.example.com) https://whatsup.linguistik.uzh.ch/01_corpus/02_preprocessing/02_without_permission?rev=1656314505&do=diff 1.2.2 Data without permission During the data collection process, not all communication partners in all the chats gave their permission for their texts to be used. In chats where we did not get the permission of all participants, we still used the messages for which we had the permission and disguised those without. text/html 2022-06-27T07:21:45+00:00 Anonymous (anonymous@undisclosed.example.com) https://whatsup.linguistik.uzh.ch/01_corpus/02_preprocessing/03_emojis?rev=1656314505&do=diff 1.2.3. Emojis Emojis are characters in Unicode. The application WhatsApp uses special fonts such as to have the same appearance of emojis on all operation systems. In our corpus browsers, emojis can be displayed, but they are represented in the font that is used by the user, thus, it cannot be guaranteed that an emoji in the original text looked as it does on your screen. text/html 2022-06-27T07:21:45+00:00 Anonymous (anonymous@undisclosed.example.com) https://whatsup.linguistik.uzh.ch/01_corpus/02_preprocessing/04_languages?rev=1656314505&do=diff 1.2.4 Languages and varieties Languages and varieties per chat In order to assign a language tagging to each chat, we looked at the first 250 messages and assigned two possible attributes per language: * lang_100_and_more: Languages that were found in more than 100 messages text/html 2022-06-27T07:21:45+00:00 Anonymous (anonymous@undisclosed.example.com) https://whatsup.linguistik.uzh.ch/01_corpus/02_preprocessing/05_technical_messages?rev=1656314505&do=diff 1.2.5 Technical messages WhatsApp sometimes generates messages when something happens in a chat. If, for example, a user leaves a group, the message can be Peter left or Peter hat die Gruppe verlassen. In order to get one wording for these messages, we encoded them, e.g. text/html 2022-06-27T07:21:45+00:00 Anonymous (anonymous@undisclosed.example.com) https://whatsup.linguistik.uzh.ch/01_corpus/02_preprocessing/06_pos?rev=1656314505&do=diff 1.2.6 Part of Speech Tagging Some sub-corpora have been annotated with Part Of Speech annotations. This concerns WUS_DIALOG_GSW, WUS_FRA, WUS_FRA_DEMOG, WUS_ITA, WUS_ITA_DEMOG. French The whole French corpus has been annotated with MElt (Modified French TreeBank) using the tag set text/html 2022-06-27T07:21:45+00:00 Anonymous (anonymous@undisclosed.example.com) https://whatsup.linguistik.uzh.ch/01_corpus/02_preprocessing/07_normalization?rev=1656314505&do=diff 1.2.7 Normalization Normalization is the task of "translating" non-standard language data into standard language. It can be performed manually or automatically with computational linguistics tools. In the case of our corpus, we have manually normalized some data in the Swiss German dialect, resulting in the corpus WUS_DIALOG_GSW (5 chats, 34,683 tokens).