Learning mathematics is one of the problems faced by students; the fact that the lesson is difficult and boring leads many students do not like it and even they avoid mathematics to learn. This study aims to analyze the difficulty of learning mathematics and the factors experienced by students in the topic of addition and subtraction of fraction in the form of stories in grade V at SD Negeri 1 (Public Elementary School) Somagede. The type and approach of this study is qualitative. Observation, interview, test, and documentation were used to collect data. The researcher took the subject of fifth grade teachers, fifth grade students, and parents of the students. The research data that has been carried out by the researcher reveals that students still have difficulty in learning mathematics in counting and memorizing formulas, especially in the material of fraction addition and subtraction in the form of stories. The researcher took the test results of the students and found that five students had difficulty in learning mathematics by getting scores in the low category. Factors that cause students to experience difficulties in learning, namely internal factors which are reasons that arise from the students themselves in the learning process and external factors are the ressons that arise from outside students which include the family and school environment. It is expected that the teacher are able to put more emphasis on mastery of concepts, skills in using learning media tools, and student characteristics in providing material for addition and subtraction of fractions in the form of stories. The researcher's suggestion to teachers is to motivate the students when teaching mathematics to overcome the learning difficulties of mathematics, so that they are interested in learning mathematics. The teachers are also expected to be able to become fun teachers to learn mathematics and provide learning facilities for students to become more enthusiastic in learning mathematics.