This chapter explores the significance of the legal conception of caste speech, the inseparability of physical caste violence and non-physical violence, and the limitations of the liberal framework of free speech/hate speech. Locating the codification of casteist speech in the context of Dalit social movements and the historic enactment of the SC/ST Act, 1989, this chapter argues that hate speech is inadequate to recognise forms of casteist speech, which has strong similarities to the category of racial speech in the American context. Racial insult caused racial stigmatisation and undermined the dignity and self-worth of members of a racial minority. These insights from Critical Race theorists, the chapter concludes, are relevant to locate casteist speech in its historical context of power relations.