New Chinese benchmark on aligning language models, looking at fairness, safety, legality, and data protection, and a unique morality dimension that integrates specific Chinese values such as harmony.
This is super interesting and while these benchmarks can be useful, I'm not sure this is the right use of 'alignment' and that data protection is what safety researchers are worried by. A lot of these are important but they tend to focus on compliance and adherence to predefined standards.
More generally, I wonder if some approaches to safety risk encouraging a narrow view of acceptable behavior, and limiting an AI's ability to handle complex, real-world ethical scenarios. More researchers should look into evaluating a model's ability to adapt to a broader spectrum of human ethics and cultural perspectives.
I can also see why some skeptics are worried over overreach and censorship. I think it's desirable that e.g. some models avoid offensive language, but I don't think all models need to, and nor do I think doing so means the model is necessarily safe or useful. Anyway interesting research, keen to hear other views too.
arxiv.org/abs/2311.06899