Very proud of the latest cohort of Data Science thesis interns at Randstad Groep Nederland. In absence of a “real” defense at the University of Amsterdam, we organized our own afternoon packed with defenses (and subsequent drinks) in our Randstad HQ in Diemen. At the end of the afternoon we were able to congratulate Roan, Anna, and Adam on a job (almost) well done!

Roan Schellingerhout presented his work on “Explainable career Path Predictions.” Roan implemented explainable deep neural nets for predicting and explaining a job seekers’ next opportunity, given their previous. He evaluated the models intrinsically, in addition to testing them (+ their explanations) with actual recruiters, and found both that models are accurate and recruiters like and understand them.

Anna Lőrincz worked on data-to-text generation, and fine-tuned a multilingual transformer model for generating benefits (salary, contract, working hours, locations) in job descriptions in both Dutch and English, given structured information (numeric, categorical, and binary variables). She found that transformers can successfully generate fluent and correct text given structured inputs, confirmed that inputs or prompts have a high impact on performance, and found that her approach beats template-based methods in textual diversity. She also found a few very funny hallucinated work locations (“pal achter centraal station in Zwaaijdijk”, was one of our favorites), and found that transformer models tend to sometimes correct output (adjusting a 3k/hour salary into a 3k monthly salary).

Finally, Adam Mehdi Arafan presented his “Double Fair-Gated Bias Mitigation Pipeline” for our Talent Recommender system, where he studied bias in multiple parts of our recsys pipeline, from re-balancing training data (to simulate both balaned and highly imbalanced scenarios), to generating additional balanced synthetic data, and re-ranking outputs. Turns out applying synthetic data does not only help in creating more fair rankers, but can also have benefits in terms of model accuracy!

All three students did great jobs, stay tuned for their thesises (and, who knows, publications? 😏)

At a conference in Utrecht I participated in a panel discussion alongside Hans de Zwart (HvA’s Centre of Expertise Applied AI), Rina Joosten-Rabou (Seedlink), Mildo van Staden (Ministerie van Binnenlandse Zaken en Koninkrijksrelaties) and Siri Beerends (SETUP).

I summarized my takeaways on LinkedIn, which I share below (in Dutch)

Fantastic news! We’ve received word the 2nd edition of our “Recommender Systems for Human Resources” (RecSys in HR) Workshop has been accepted to be included in the ACM RecSys 2022 program, to be held in Seattle!

Last year’s (first) edition of our workshop was co-located with ACM RecSys 2021 in Amsterdam, and featured two keynotes, a panel, breakout sessions and 8 paper presentations. The recording, workshop proceedings, and a workshop report are available through our workshop’s website at: https://recsyshr2021.aau.dk/

Check back there soon for information on the 2022 edition we’re planning with Toine Bogers, Mesut Kaya, Francisco Gutiérrez, and newly joined co-organizers Sepideh Mesbah (Randstad Groep Nederland) and Chris Johnson (Indeed.com)!

I’m giving a talk at the KINTalks series, organized by the KIN Center for Digital Innovation on March 25. It’s going to be a hybrid event, so happy to meet you at the VU Amsterdam, and if not, see you online! RSVP here on EventBrite.

KINTalks is a hybrid event where practitioners are invited to talk about their work experience regarding innovation and digital technology.

The blurb

At Randstad, the global leader in the HR services industry, searching and matching is at the heart of what we do. Being founded in 1960, We know from our heritage that real connections are not made from data and algorithms alone – they require human involvement. Last year, we helped more than two million job seekers find a meaningful job by combining industry-scale recommender and search systems with our distinct human touch. While many opportunities exist, employing AI in recruitment and HR is considered high-risk by the European Commission’s proposed regulatory framework on AI, which will bring additional requirements, obligations, and constraints.

In this hybrid talk, I will explain some of the characteristics of, challenges, and opportunities in the HR domain from an AI perspective. I will share some of our own work in recommendations, algorithmic matching, algorithmic bias and knowledge graphs, and highlight some of the ongoing research in this domain.

We have two papers accepted at “The First International Workshop on Computational Jobs Marketplace“, co-located with WSDM 2022. Both papers are based on work done by two of our former thesis interns at Randstad Groep Nederland!

• N. Vermeer, V. Provatorova, D. Graus, T. Rajapakse, and S. Mesbah, “Using robbert and extreme multi-label classification to extract implicit and explicit skills from dutch job descriptions,” in Compjobs ’22: the first international workshop on computational jobs marketplace, 2022.
  [Bibtex]

  @inproceedings{vermeer2022using,
  author = {Vermeer, Ninande and Provatorova, Vera and Graus, David and Rajapakse, Thilina and Mesbah, Sepideh},
  title = {Using RobBERT and eXtreme Multi-Label Classification to Extract Implicit and Explicit Skills From Dutch Job Descriptions},
  year = {2022},
  booktitle = {CompJobs '22: The First International Workshop on Computational Jobs Marketplace},
  numpages = {5},
  location = {Online},
  month={2}
  }

☝️ Ninande Vermeer worked under supervision of Sepideh Mesbah and Vera Provatorova (UvA) on: “Using RobBERT and eXtreme Multi-Label Classification to Extract Implicit and Explicit Skills From Dutch Job Descriptions” in which we study to what extent a RobBERT-XMLC model can be used to extract explicit and implicit skills from Dutch job descriptions.

• S. van Els, D. Graus, and E. Beauxis-Aussalet, “Improving fairness assessments with synthetic data: a practical use case with a recommender system for human resources,” in Compjobs ’22: the first international workshop on computational jobs marketplace, 2022.
  [Bibtex]

  @inproceedings{vanels2022improving,
  author = {van Els, Sarah-Jane and Graus, David and Beauxis-Aussalet, Emma},
  title = {Improving Fairness Assessments with Synthetic Data: a Practical Use Case with a Recommender System for Human Resources},
  year = {2022},
  booktitle = {CompJobs '22: The First International Workshop on Computational Jobs Marketplace},
  numpages = {5},
  location = {Online},
  month={2}
  }

✌️ Sarah-Jane van Els worked under supervision of myself and Emma Beauxis-Aussalet (Civic AI Lab) on “Improving Fairness Assessments with Synthetic Data: a Practical Use Case with a Recommender System for Human Resources” in which we explore approaches and methods for assessing algorithmic bias by using synthetic data to improve the size and representativity of a test set used for training candidate recommender systems.

👏 Proud of our former interns for having published their work! And happy with the collaborations we have had with our co-authors 😁.

Update: unfortunately but understandably (in light of the pandemic), this conference has been post-poned until further notice.

December 9th I will join a panel discussion on AI and equal opportunities in the labor market (together with Siri Beerends & Rina Joosten-Rabou), at a conference organized by WOMEN INC. An offline event in Utrecht (Corona volente)!

More details and RSVP (in Dutch) here: Congres | “Hey Siri: Vind een geschikte kandidaat”

For those who missed it — like myself 😏 — we have now published the full recording of our Recommender Systems in Human Resources (RecSys in HR) Workshop, which was held on October 1st, in conjunction with the ACM RecSys Conference in Amsterdam.

Our workshop included two keynotes, eight paper presentations, breakout sessions, a virtual panel on the topics of the upcoming EU framework on AI, Fair & inclusive HR Tech, and how to “activate hidden workers.” See the full program here.

So, if you have 4h33m to spare, see the full recording below! 

(or use the convenient YouTube chapter to jump through the program 😅)

Many thanks to all co-organizers, contributors, paper authors, and participants, both virtual and in-person! And hopefully we’ll see each other at the second edition ✌️.

Another academic year, another (short)list of potential projects. Are you a final-year AI or data science student, interested in doing an internship with us? Reach out! First, read below why you would want to join us, and scroll further down for the following project descriptions:

1. Job Description Generation (NLP)
2. Conversational/QA approaches for resume information extraction (NLP)
3. Segmentation of resumes (NLP, CV)
4. Synthetic data for bias mitigation in recommender systems
5. Career pathing
6. Algorithmic planning
7. Open project: IR, RecSys, NLP, or fair AI

Together with Helen Hulsker I joined of a webinar on governing AI by Dataiku, read the blurb here:

There are many obstacles for Data Scientists in effectively governing the use of AI such as, overcoming the bias inherent in historical data, agreeing on the boundaries of governing AI with the business, and perhaps most importantly, ensuring the adoption of good AI practices across the organisation.

So join Randstad and Dataiku as we set the stage around implementing AI in a high-risk domain from a technical/practical and jurisdictional perspective.

Check out the recording at BrightTALK!

Excited to be giving a keynote at the International Workshop on Fair, Effective And Sustainable Talent management using data science (FEAST workshop), part of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2021).

I’ll give a talk about some of our recent data science projects Randstad Groep Nederland, spanning from our job/candidate recommender systems, work on bias and bias mitigation, knowledge graphs, and deep embeddings for learning to match candidates to vacancies.