
Defense of the dissertation of Brimzhanova Saule Serikovna for the degree of Doctor of Philosophy (PhD) in the specialty «6D060200 - Computer Science»
L.N. Gumilyov Eurasian National University, a dissertation defense for the degree of Doctor of Philosophy (PhD) by Brimzhanova Saule Serikovna on the topic «Development of an intelligent system for assessing and controlling knowledge using the algorithm of shingles» by specialty «6D060200 – Computer Science».
The dissertation was completed at the department of «Information Systems» of Akhmet Baitursynuly Kostanay Regional University.
The language of defense is russian
Official reviewers:
Iklasova Kainizhamal Esimseitovna - Doctor of Philosophy (PhD) in specialty «Informatics, Computer Science and Management», Associate Professor of the Department of «Information and Communication Technologies», M. Kozybaev North Kazakhstan University (Petropavlovsk, Republic of Kazakhstan);
Akanova Akerke Saparovna - Doctor of Philosophy (PhD) in specialty «Informatics, Computer Science and Management», acting associate professor of the Department of «Computer Science», Faculty of Computer Systems and Professional Education, S.Seifullin Kazakh Agro-Technical Research University (Astana, Republic of Kazakhstan).
Temporary members of the Dissertation Council:
Mukhanova Asel Serikovna - Doctor of Philosophy (PhD) in the specialty “6D060200 – Computer science”, Dean of the Higher School of Information Technology and Engineering at the Astana International University (Astana, Republic of Kazakhstan);
Mamyrbayev Orken - Doctor of Philosophy (PhD), Associate Professor, Deputy General Director in science of the Institute of Information and Computing Technologies (Almaty, Republic of Kazakhstan);
Luigi La Spada - PhD, Associate Professor in the Department of "Electrical and Electronic Engineering," Edinburgh Napier University (Edinburgh, Scotland, United Kingdom).
Scientific advisors:
Atanov Sabyrzhan Kubeisinovich - Doctor of Technical Sciences, Professor of the Department of “Computer and Software Engineering”, L.N. Gumilyov Eurasian National University (Astana, Republic of Kazakhstan);
Moldamurat Khuralay - Candidate of Technical Sciences, Associate Professor of the Department of “Space Engineering and Technology”, L.N. Gumilyov Eurasian National University (Astana, Republic of Kazakhstan);
Gagarina Larisa Gennadyevna - Doctor of Technical Sciences, Professor, Director of the Institute of System and Software Engineering and Information Technologies, National Research University “Moscow Institute of Electronic Engineering” (Zelenograd, Russian Federation).
The defense will take place on December 03, 2024, at 11:00 AM in the Dissertation Council for the training direction «8D061 – Information and communication technologies» in the specialty «6D060200 – Computer Science» of L.N. Gumilyov Eurasian National University. Conducting a meeting of the dissertation council in a hybrid (offline and online) format.
Address: Astana, K. Satpayev str. 2, Main building, room No. 302.
Abstract (English): ANNOTATION dissertation work of Brimzhanova Saule Serikovna on a theme «Development of an intelligent system for assessing and controlling knowledge using the algorithm of shingles», submitted on competition of a degree of the doctor of philosophy (PhD) on a specialty «6D060200 - Computer Science» Relevance of the thesis topic. The relevance of the dissertation research on the topic «Development of an intelligent system for assessing and controlling knowledge using the algorithm of shingles» is determined by the following key factors: 1. Labor Market Transformation: The modern labor market is undergoing significant changes due to the rapid development of innovative technologies and new economic segments. This leads to a constant complication of professional and qualification requirements. Under such conditions, effective assessment and control of knowledge become critical for successful adaptation and development in the labor market. Intelligent systems that promptly and accurately assess the qualifications of workers contribute to their successful integration into the changing conditions of the labor market. 2. Growth of distance learning: Significant spread of distance learning and active use of information and communication technologies in the educational process revealed insufficient effectiveness of the existing systems of knowledge control. The results showed that traditional approaches do not always meet the requirements of the remote learning format. This emphasizes the need to develop new intelligent systems that can improve the quality of the educational process and adapt to the peculiarities of distance learning. 3. The need to improve approaches to knowledge assessment: Current methods of knowledge assessment do not always provide uniformity and accuracy, especially in humanities disciplines, where deep analysis of texts is required. The development of an intelligent system capable of implementing a shingle algorithm and taking synonyms into account to more accurately determine the similarity of texts will overcome current difficulties and improve the accuracy of assessment. This is especially important for testing, where the evaluation of extended answers and identification of originality of texts is required. 4. Normative-legal support: The relevance of the research is confirmed by a number of normative-legal acts of the Republic of Kazakhstan. In particular, the Address of the Head of State to the people of Kazakhstan dated September 1, 2023 focuses on the need for digitalization of the educational system and introduction of advanced technologies in the educational process. The Law of the Republic of Kazakhstan “On Education” (as amended on July 5, 2024) emphasizes the importance of integrating innovative technologies, including intelligent knowledge assessment systems, into educational practice. Also, new regulations governing digital technologies and artificial intelligence promote the development and application of intelligent systems in distance learning that meet modern educational standards. Thus, the development of an intelligent system of knowledge assessment and control using the shingles algorithm is a highly relevant task. It is aimed at optimizing the process of knowledge assessment, improving the quality of learning and increasing the accessibility of education in the context of a dynamically changing labor market and the growing use of information and communication technologies. The aim of this thesis is to develop an intelligent system that uses the shingles algorithm to assess knowledge through open-ended testing, ensuring the reliability and security of the results. Research Objectives: 1.To study the algorithm of shingles for the purpose of application in the intelligent system of open testing; 2.Identify ways to determine the correctness of students' answers using odd logic; 3.Develop an intellectual system of open testing using the algorithm of shingles and odd logic, providing reliability and security of the results; 4.Compare the result of the intellectual system of open testing, taking into account the time of providing answers by students and the complexity of questions with the teacher's assessment. The object of the dissertation research is the process of assessment and control of students' knowledge in the conditions of open testing using intelligent systems based on the algorithm of shingles and fuzzy logic. The subject of the dissertation research is the development and application of an intelligent system using the shingles algorithm and fuzzy logic for the assessment and control of knowledge in order to improve the reliability and security of the results of open testing. Research Methods: The thesis work used programming methods to implement the system, analytical research to analyze the shingles algorithm and fuzzy logic, set theory of mathematics to develop models, system analysis to evaluate the functionality of the system, group expert judgment design to integrate expert opinions, and structural and semantic modeling to create and analyze models. Experiments were set up using the mathematical packages FUZZY LOGIC TOOLBOX and MATLAB. Theoretical basis: For a full understanding of the problem under study it is necessary to refer to the works of such scientists as Kabdulova G. A., Dulatov S. Zh. J., Mukashev A. S., Basilkhanov A. K., Karabaev K. K., Kazhigalieva M. E., Musagaliev J. A., Sarmurzina J. K., and Tusupbekov K. K., Tusupbekov S. A., Tonysheva T. S., Aldabergenova A. S., Kazybek N. E., Makarova S. I., Kulikov D. P., Danilova S. D., and foreign researchers. D., as well as foreign researchers: Bride H., Tillmann N., Blaylock, Ly, Jan-Eric Eriksson, Shirley Alexander, Charles Duncan and Dave Closser. Important contributions to the development of intelligent systems and knowledge assessment have also been made by Kazakh scientists, such as A. A. Sharipbaev, N. B. Seitov, A. M. Baimagambetov, A. T. Nurpeisov, N. Kaliev, N. K. K., Musin A. S., Ibraimov N. A., Tursynbekov R. K., Kamalidinov A. B., and Kosarev A. S. These works provide the necessary theoretical background. S. These works provide the necessary theoretical basis and context for the development of an intelligent system of knowledge assessment and control using the shingles algorithm. Main results and scientific novelty of the study: 1. Result: The study of shingles algorithm for the purpose of application in intelligent open-ended testing system. Novelty: The study of the application of the shingles algorithm for adaptation in the intelligent system of open testing, which can improve the accuracy of automatic evaluation of answers. 2.Result: Identification of ways to determine the correctness of students' answers using odd logic. Novelty: The methods of using odd logic to evaluate the correctness of answers, taking into account the variability of wording and synonyms, which improves the objectivity and accuracy of evaluation, are developed and implemented. 3. Result: Development of an intelligent system of open-ended testing using the algorithm of shingles and odd logic, providing reliability and security of results. Novelty: An intelligent open-ended testing system is developed that combines the shingles algorithm and fuzzy logic to improve the reliability and security of assessments, which minimizes the risks of errors in automatic knowledge assessment. 4.Result: Comparison of the performance of the intelligent open-ended testing system considering the time of providing answers by students and the complexity of questions with the teacher's evaluation. Novelty: A comparative analysis of the effectiveness of the intelligent system of open testing in comparison with the traditional assessment of the teacher, taking into account the time cost and complexity of questions, which allows us to assess the performance and accuracy of the system. These results have considerable novelty and broad applicability in a variety of educational contexts. They make a significant contribution to the development of intelligent assessment and knowledge control systems, significantly enriching and improving existing approaches and methods. Practical significance of the obtained results. The practical significance of the obtained results lies in their direct application in educational institutions to improve the accuracy and objectivity of students' knowledge assessment. The developed intellectual system of open testing, using the algorithm of shingles and fuzzy logic, is able to significantly improve the process of knowledge assessment. Specific benefits include: 1.Improved assessment accuracy: The system provides a more objective and accurate assessment of knowledge by minimizing subjective factors and reducing the possibility of errors that can occur in traditional testing methods. 2.Versatility of application: The developed system can be easily adapted for use in educational programs and at different levels of learning, making it useful for a wide range of educational organizations. 3.Optimization of the testing process: Implementation of the system allows automating and speeding up the assessment process, freeing up teachers' time for more productive work and providing students with timely feedback. 4.Strengthening data protection: The system implements advanced mechanisms to protect test results from tampering and unauthorized access, which is of particular importance in the digitalization of education. Including extensive use of electronic signatures, encryption keys and other authentication methods, the system provides a high level of security and trust in test results, which is especially important in the context of distance learning and e-testing. These aspects make the developed system not only innovative, but also practically significant for improving the quality of education in the context of modern digitalization. Economic significance of the results obtained. The economic significance of the obtained results lies in their potential to significantly reduce the costs associated with the process of knowledge assessment. The developed intelligent open testing system based on shingles algorithm and fuzzy logic can provide the following economic benefits: 1.Reduction in resource costs: Automation of the testing process reduces the need for a significant number of human resources for test checking and evaluation of tests, which reduces labor costs and allows more efficient use of resources of educational institutions. 2.Reduced assessment time: The system speeds up the assessment process to provide results and feedback to learners more quickly, thus improving the overall efficiency of the learning process. 3.Reduced Error Correction Costs: Improving the accuracy and reliability of automatic assessment reduces the number of errors and the need for re-checking, which also helps to reduce the associated costs. 4.Increased accessibility and scalability: The scalability of the system across different educational institutions and programs makes it accessible to a wide audience, increasing its cost-effectiveness and the feasibility of the investment in system development and implementation. Thus, the developed system not only improves the quality of the educational process, but also provides significant economic benefits, reducing costs and increasing the efficiency of educational institutions. The main points of the thesis to be defended are: 1.An intelligent knowledge evaluation system using shingles algorithm and fuzzy logic for open testing has been developed and tested. The system provides high reliability and security of results due to hashing of shingles obtained from canonicalization and semantic analysis of text. 2.The system demonstrates high accuracy of knowledge assessment, which makes it a promising tool for controlling learners' knowledge and confirms its effectiveness and validity in real-world conditions. 3.Approbation of the system in real educational conditions confirmed its effectiveness and practical value, which emphasizes the significance of the study. 4.The results are confirmed by author's certificates, which indicates the significance of the development and its potential for application in education and other fields. Approbation of the research results. The main results of the thesis work were presented and discussed at a number of scientific and professional events. The list of these events includes: 1.The 7th International Conference on Digital Technologies in Education, Science and Industry DTESI-2022, which was held in Almaty, Kazakhstan, October 20-21, 2022. 2.Institute of Systems and Software Engineering and Information Technology Workshop at the National Research University of Moscow Institute of Electronic Engineering in Zelenograd, Russia, June 15, 2019. 3.16th International Conference on Electronics, Computers and Computing ICECCO-2021 held at Almaty, Kazakhstan in the year 2021. 4.6th International Research and Practice Conference: managing innovation and technology in the era of globalization, which was held in Pattaya, Thailand on January 9-11, 2019. 5.International Conference: science through the eyes of youth, which was held in Kostanai, Kazakhstan, May 24, 2019. 6.The 5th International Scientific and Practical Conference: science and education in the modern world, which was held in the city of Nur-Sultan, Kazakhstan, December 10-12, 2019. 7.6th International Scientific and Practical Conference: science and education in the modern world, which was held in Nur-Sultan city, Kazakhstan, in April 2020. 8.2nd International Scientific and Practical Conference: development of military education in the context of ensuring military security of Kazakhstan, which was held in the city of Petropavlovsk, Kazakhstan, in 2019. 9.5th International Conference on Engineering and MIS 2019, which was held in Astana, Kazakhstan, on June 6-8, 2019. 10.International Conference on Intelligent Information Systems and Technologies - SIST 2022, which was held in the city of Nur-Sultan, Kazakhstan, April 28-30, 2022. 11.Round table on the topic Social policy of the state: new ways in the development of Kazakhstan in Kostanai Academy of the Ministry of Internal Affairs of the Republic of Kazakhstan named after Sh. Kabylbaev, which was held in the city of Kostanai, Kazakhstan, February 18, 2020. 12.Meetings of the Department of Information Systems and Informatics in KRU named after A.Baitursynov. 13.Seminars of the Department of Information Systems and Informatics in “KRU named after A. Baitursynov” in December 2019. 14.Seminars in OSH DUIS in Kostanay region, as well as in children's educational centers in order to identify personal qualities of employees. Reports and presentations made at these events provided feedback from the scientific community and practitioners, which contributed to further improvement and development of the research results presented in the thesis. - Completeness of reflection of the dissertation results in publications: - Публикации in the editions included in the scientometric databases Web of Science and Scopus: 1. Brimzhanova Saule, Atanov Sabyrzhan, Moldamurat Khuralay // An intelligent testing system development based on the shingle algorithm for assessing humanities students' academic achievements // Education and information technologies. – 2022. – Vol. 27, Iss. 8. – P. 10785- 10807; - Publications in the collections of conferences included in the Scopus database: 1.Brimzhanova S., Atanov S., Moldamurat Kh. (2019). // Problems of detecting fuzzy duplicates // ACM International Conference Proceeding Series, doi:10.1145/3330431.3330455; 2.Brimzhanova S., Atanov S., Moldamurat Kh. (2019). // Cross-platform compilation of programming language golang for Raspberry Pi. // ACM Proceeding Series International Conference, doi:10.1145/3330431.3330441; - Publications in publications recommended by the Committee for Quality Assurance in Science and Higher Education of the Ministry of Science and Higher Education of the Republic of Kazakhstan: 1.Brimzhanova S.S., Atanov S.K., Gagarina L.G., Tashatov N.N. (2019) // Shingles algorithm - a method of fuzzy comparison of lines // Bulletin of the Kazakh National Research Technical University named after K. I. Satpayev, No. 3 (133), P. 311-317. ISSN 1680-9211; 2.Brimzhanova S.S., Atanov S.K., Moldamurat H. (2020) // Intellectualy zhuyeni bagalau zhane bilimdi bakylau componenti // Bulletin of the Kazakh National Technical University named after K.I. Satpayev. Almaty: Izd-vo KazNITU named after K.I. Satpayev, № 5 (141), p.243-247. ISSN 1680-9211; 3.Brimzhanova S.S., Atanov S.K., Moldamurat H. // Implementation of an intelligent testing system application // Scientific and Technical Journal Vestnik AUES, № 2 (49), 2020. С. 122-128. ISSN 1999-9801; - Publications in materials of international and republican scientific-practical and scientific-theoretical conferences: 1.Brimzhanova, S. S. (2019) Software testing // Atanov S. K., Gagarina L. G. // Innovation Management and Technology in the Era of Globalization: Materials of the VI International Scientific-Practical Conference. January 9-11, 2019, Pattaya, Thailand: Regional Academy of Management, p. 184-189, ISBN 978-601-267-338-8; 2.Brimzhanova, S. S. (2019) Intelligent system // Atanov S.K. // Proceedings of the International Conference: Science through the eyes of youth, dedicated to the year of youth, Kostanay Academy of MIA RK named after Sh. Kabylbaev, May 24, 2019, p.397-401, ISBN 978-601-210-291-8; 3.Brimzhanova, S. S. (2019) // Intellectualyқ akparattyқ zhuyelerdің мәселелері // Proceedings of the 5th International Scientific and Practical Conference (Technical Sciences, I Volume) - Nur-Sultan, 2019 - 46-50 p, ISBN 978-601-332-366-4; 4.Brimzhanova S.S. (2020) Testileu bilim alushylardyң bilim sapasyn bakylaudynң objectivti әdіsі retinde // Atanov S.K. // Science and Education in the Modern World / Proceedings of the 6th International Scientific and Practical Conference, I Vol. - Nur-Sultan, 2020 - 36-41 p. ISBN 978-601-332-271-1; - Copyright Certificates (Appendix A): 1.Brimzhanova S.S. // Data encryption program using Rijndael algorithm (11/03/2019) - program for computer, №2812 of April 15, 2019 MJ RK; 2.Brimzhanova S.S., Atanov S.K. // Intelligent testing system using Shingles algorithm - program for computer, № 10973 of June 18, 2020 Republic of Kazakhstan; 3.Brimzhanova S.S., Atanov S.K., Moldamurat H. // Synonymi bar modificationlangan shingle algorithm arkyly bilim alushynynynң bilimin jasandy intellect bagalau bagdarlamasy - program for computer, № 33600 from March 15, 2023 the Republic of Kazakhstan; 4.Brimzhanova S.S., Atanov S.K., Moldamurat H. // Program for normalization of sentences with keywords selection for system of intellectual testing. // program for computer, № 39118 from “19” September 2023. - Act of Implementation (Appendix B): 1.Act of introduction of the software Intellectual testing system with the use of the algorithm of shingles in the educational process of the department of organization of social work in the internal affairs bodies of the faculty of professional training of the Kostanay Academy of the Ministry of Internal Affairs of the Republic of Kazakhstan named after Shrakbek Kabylbaev from September 30, 2022; 2.Act of introduction of the software “Intelligent testing system using the algorithm of shingles” in the educational process of the Academy of National Guard of the Republic of Kazakhstan from November 8, 2023; On the topic of the dissertation research 10 scientific papers on research work have been published. Of these, 1 article in the journal database Scopus and Web of Sciences (Q1, percentile 93), 2 articles in the conference proceedings included in the Scopus database, 3 articles in journals recommended by the Committee for Quality Assurance in Science and Higher Education MNVO RK, 4 articles in the journals of international and national scientific-practical and scientific-theoretical conferences, 4 author's certificates of computer programs, 2 acts of implementation of research results. Compliance with the directions of scientific development or government programs: 4. Information, communication and space technologies. 4.1 Artificial intelligence and information technology. 4.1.3 Pattern recognition and image processing; 4.1.5 Machine learning; 4.5 Methods and systems of information security and data protection. 4.5.1 Methods and algorithms for ensuring information security of complex systems and data; 4.5.2 Technologies and software and hardware for information protection. 9. National security and defense. 9.2 Applied scientific research. 9.2.1 Ensuring information security. Personal contribution of the co-researcher to each publication. The co-researcher made a significant personal contribution to the development and approbation of the intelligent system of knowledge assessment, including conducting theoretical analysis, creating and modifying shingles algorithms, designing the system architecture, organizing and conducting experiments. The results of the research were published in scientific articles and presented at round tables, conferences, seminars and department meetings. The author of the dissertation is either the first or corresponding author in all these articles, which confirms his direct and full participation in the conducted research. He also initiated the obtaining of author's certificates confirming the novelty and innovativeness of the development. The structure and scope of the dissertation. The dissertation consists of an introductory part, a main part (three sections), a conclusion, a list of sources used and an appendix. In the introduction, the choice of the research topic is justified, the main components of the scientific research methodology are described, the relevance of the research is argued, the degree of its study in practical and theoretical significance is formulated, the purpose of scientific research is formulated, the tasks of the dissertation work are defined, the object and subject of the work are presented, scientific novelty is disclosed, the main provisions submitted for protection, the personal contribution of the author, the list of scientific publications and approbation of the results of the dissertation research. Section 1 of the dissertation is devoted to the definition of the basic concepts used in modern education, the analysis of the prerequisites of the research, the formulation of the problem and the justification of the significance of this research. The section also highlights the limitations and delineations, the assumptions of the study, as well as the theoretical and conceptual foundations. The development of the digital economy and the global transformational changes taking place in the labor market create new challenges for the higher education system. Employers are setting new criteria for assessing the level of professional competence of future specialists. In such conditions, personnel with a wide range of soft skills, or so-called soft skills, become important and competitive. These include communication skills, a high level of self-regulation, critical thinking, professional and labor mobility, leadership and creativity. As part of the study, an analysis of modern testing systems was carried out, which are used to assess the knowledge and skills of students. This analysis made it possible to identify the features and disadvantages of existing systems and prepare the basis for the development of an intelligent knowledge assessment and control system using the Shingle algorithm. As a result of this research, a model of an intelligent knowledge assessment and control system was developed, as well as a modified shingle algorithm for its integration. The proposed method of increasing the effectiveness of the introduction of open testing, as a progressive method of knowledge control for students of the humanities, was also considered and justified. In addition, the architecture of an automated intelligent system using the Shingle algorithm was developed. Thus, section 1 of the dissertation provides an overview of the main theoretical and practical aspects of the study, based on an analysis of modern labor market requirements and existing testing systems. In section 2 of the dissertation, based on the analysis of scientific literature, descriptions of approaches to the definition and understanding of artificial intelligence are presented, as well as an overview of current research directions in this field. During the study, an analysis of the test results was carried out, which included an assessment of the effectiveness and functionality of the intelligent system. The peculiarities of the operation of this system, including its capabilities and limitations, were revealed. The analysis of the test results allowed us to draw conclusions about the efficiency and effectiveness of the developed intelligent knowledge assessment and control system using the Shingle algorithm. Based on this analysis, the advantages and advantages of the system were highlighted, such as the accuracy of knowledge assessment, ease of use and automation of the assessment process. Its features were also considered and discussed, for example, the ability to process large amounts of information and work with various types of tasks. As a result of the study, positive results were obtained confirming the efficiency and effectiveness of an intelligent knowledge assessment and control system using the Shingle algorithm. These results represent an important contribution to the field of artificial intelligence and its application in educational processes. Section 3 of the dissertation presents the algorithms and methods used in the intellectual system of assessment and control of knowledge. These algorithms and methods have been developed to ensure effective and accurate assessment of students' knowledge. As part of the study, the effectiveness of the developed algorithm was tested among students of 2-3 courses in the humanities, such as future teachers, lawyers and psychologists. The selection of this sample was carried out on the basis of the criterion of training in humanitarian educational programs, in which open-ended questions are required during testing. The experiment included testing using the developed algorithm among the specified group of students. During the test, the effectiveness and accuracy of the algorithm in assessing the knowledge of students of the humanities were evaluated. Various parameters and factors were taken into account, such as the correctness and completeness of the answers, the completion time of the tasks and the quality of the assessment. The results of the effectiveness check allowed us to draw conclusions about the suitability of the developed algorithm for use in an intelligent system for assessing and monitoring the knowledge of students of the humanities. The algorithm has shown high accuracy in assessing knowledge and the ability to process open-ended questions, which is an important criterion for testing in this area. Thus, the developed algorithm has successfully passed the effectiveness test among students of the humanities, which confirms its potential and applicability in an intelligent knowledge assessment and control system. This is a significant result of the research, which is important for the development of educational processes and the field of education. At the end of the dissertation work, the key results of the study were summarized and the main conclusions were formulated, which confirm the reliability of the provisions submitted for defense. An intelligent system using the Shingle algorithm and fuzzy logic to evaluate knowledge through open testing has been developed and successfully tested. This system provides high reliability and security of the results due to hashing of shingles obtained in the process of canonicalization and semantic analysis of the text. The results demonstrate high accuracy in assessing students' knowledge, which makes the system a promising tool for knowledge control. Testing of the system in real conditions has confirmed its effectiveness and reliability. The results obtained are confirmed by the author's evidence, emphasizing the importance of the study and its potential value for practical application in education and other fields. The appendix contains author's certificates, certificates of implementation and implementation, program codes.
