AI is rapidly evolving, reshaping our daily interactions and redefining industries, employment, and societal practices. Yet, amid this progress, it's crucial to temper the pace and examine issues like privacy, bias, and misinformation that disproportionately affect vulnerable groups – women, children, persons with disabilities, and racial minorities. Through the Human Rights lens, the ARP analyses how AI impacts these groups' rights, discusses the main trade-offs at play, and brings some proposals on regulating AI for Vulnerable Groups.