This article aims to formulate the relation between culture and society from the perspective of Niklas Luhmann’s theory of self-referential social systems. Traditional sociological theories, such as those proposed by Talcott Parsons and Alfred Schutz, consider the social to be based on a common culture among people. This idea posits that a society must be defined as a society of culture because it is a product of culture. Conversely, the theory of self-referential social systems does not presuppose such a common basis for the social. Even without a shared culture, a social system always emerges based on mutual intransparency (i. e., double contingency among people) and subsequently begins to control itself by remembering and forgetting the memory of its own operations. This memory is called culture, and it results from self-referential system operations. Therefore, as a product of the social system, culture can be described as a culture of a social system . This formulation is also applicable to the de-nationalized and de-territorialized world society. The world culture, or the culture of society , means contingency. As the eigenvalue of the world society, contingency gives the orientation of the further system-differentiation to the world society. Consequently, by placing the social before the cultural, a theoretical framework can be established for describing and analyzing the current world situation, in which cultures are being differentiated with many associated issues.