Thesis Landscapes from the Dark Alleys of Turkish Academy
The author‘s note: This is the English translation of the article that was originally published here in Turkish. Dr. Kaan Öztürk and Pınar Çelik helped me with the translation. I would like to thank Dr. Kaan Öztürk specifically for his significant contribution to the translation. I also thank Anoush Dadian and Dr. Debora Weber-Wulff for proofreading. Since this article was written for scientists from Turkey, most of the resources referred from within the article are also in Turkish. I am aware that they will not be useful for most of the readers, but I wanted to keep them in the translated version anyway for the sake of the integrity of the translation. I hope this work would be beneficial for scientists who would like to develop an opinion about the concurrent issues in Turkey. The abbreviation ‘YÖK’ appears many times in the text stands for The Council of Higher Education in Turkish, a governmental agency, the purpose of which is to ‘regulate’ the science and scientists in Turkey to meet the expectations of the authority. Please don’t hesitate to get in touch if you have any questions.
From budget issues to the government’s influence on science, the lack of free speech to the abundance of plagiarism, science in Turkey suffers on many fronts, and the academics seem to have no power to change its course.
But who is responsible for the bottleneck in the academy in Turkey?
I believe that this is a difficult question to answer given its multi-dimensional attributes, but it is a very critical question that needs to be addressed thoroughly. This article does not offer a clear answer to this question, but instead, offers an exploratory approach examining the major issues.
It is evident that the academic community in Turkey needs a mind exercise on this subject more than the answer itself.
Table of contents / İçindekiler
- 1 Introduction
- 2 Previously Published Material on Academic Problems in Turkey
- 3 Thesis Landscapes from Turkey: Students and Advisors
- 3.1 Advisor: Dr. Güniz Gürüz, METU
- 3.2 Advisor: Dr. Sinan Bilikmen, METU
- 3.3 Advisor: Dr. Bekir Karlık, Fatih Üniversitesi
- 3.4 Advisor: Dr. Bekir Karlık, Sakarya Üniversitesi
- 3.5 Advisor: Dr. Halit Pastacı, Yıldız Teknik Üniversitesi
- 3.6 Advisor: Dr. Ali Okatan, Haliç Üniversitesi
- 3.7 Advisor: Dr. Ali Okatan, Bahçeşehir Üniversitesi
- 3.8 Advisor: Dr. İsmail Naci Cangül, Uludağ Üniversitesi / Balıkesir Üniversitesi
- 3.9 Advisor: Dr. Erdem Uçar, Trakya Üniversitesi
- 3.10 Advisor: Dr. Cevdet Emin Ekinci, Fırat Üniversitesi
- 4 It is Difficult to Access Theses Published in Turkey
- 5 No Clear Reaction to Scientific Theft in Turkey
- 6 Conclusion
- 7 The Author
Academy breeds its next generation at any given time. Being self-financing in the context of “human resources”, it differs from many other professional fields. One of the disadvantages of this property is the fact that bad academy generates more bad academy. With this respect, it wouldn’t be too much to assume that it is the previous generations who are responsible for the unqualified majority of the scientists and their science in today’s Turkey. Moreover, the next generation of the academy will not be any more qualified than the current generation, unless a radical transformation takes place.
I find the role of the parents in a society very similar to the role of thesis advisors and thesis committees in higher education. To clarify this idea, I will use examples of theses that were signed by thesis advisors and passed thesis committees in Turkey to exemplify how the ethical mentality of advisors influence the understanding of the ethics of the people they graduate.
Just as a child alone cannot be blamed for its problematic behavior, ethical problems in a thesis cannot be attributed solely to the failure of the Ph.D. student.
This article is composed of several parts related to each other. The first part is the part that refers to the previous publications pertaining to academic problems in Turkey. The second part offers examples from theses which present ethical problems of varying degrees. The third part refers to the difficulty of accessing theses published in Turkey. The fourth part addresses the insufficient responses given to the academical problems in Turkey, as well as legislative and executive problems. The last part includes a short summary of the article.
There are some points I want to clarify before starting: My goal is not accuse anyone in this article. Some of the people referred to in the article are in fact the people who suffered from the things that are being discussed in this article. Therefore, it would be a mistake to place blame on the people whose names appear throughout the text. Please do not disregard the fact that you do not know the whole story, since the article does not include the opinions of others. Do not give up skepticism by relying on the author.
You are being given access to most of the theses and reports included in the article by means of dedicated efforts of Dr. Tansu Küçüköncü and various anonymous individuals. I acknowledge Dr. Tansu Küçüköncü and anonymous academicians for their careful studies and determinations, and the anonymous auditors who inspected three of the theses. I also thank Serkan Köybaşı for his support in legal aspects. Finally, I appreciate everyone who spent their time to read this article and put an effort towards ensuring that these discussions reach larger groups.
Previously Published Material on Academic Problems in Turkey
Turkey has a remarkable collection of written material on the subject of academic deterioration.
It is impossible to provide references for each and every one of them, or expect someone to read all of them. On the other hand, it is important to evaluate the problem as a whole and avoid being restricted to a single perspective. I think following articles are important as they shed light on the concurrent problems of science in Turkey from different angles (unfortunately all of them are in Turkish, but here is the list with English titles to give you a brief idea):
- Grey Areas of Scientific Immorality (A. Murat Eren)
- [National] Conferences in which Science is Being Murdered (A. Murat Eren)
- Journals of Graduate Schools in Social Sciences Publishing (Emrah Göker)
- Secret for High Numbers of Publications [from Turkey] (Metin Balcı)
- No Scientific Research is Being Done in Turkey, No Scientific Publications Either (A. Rasim Küçükusta)
- Shady Journals and Violations of Publication Ethic (Kaan Öztürk)
- Cases of Cheating in Universities and the Disciplinary Legislation – III (Levent Sevgi)
- Introduction to the Socio-Analysis on “Provincial University” (Emrah Göker)
- Conditions and Relations of Research Assistants in Provincial Universities (Emrah Göker)
- Undergraduate Courses in Provincial Universities (Emrah Göker)
- Look What Science Does to Science: Two Scandals (Orhan Bursalı)
- Mobbing, Universities and Social Sexism (Özlem Sert, Arzu Akkoyunlu Wigley)
- May Universities Avoid from Custody? (Ayşe Durakbaşa)
- Scientists are Being Attacked by the Government (Neşe Özgen (An Interview of Işıl Öz))
- “What Turkey Needs is an Academic Revolution” (Lale Akarun, Mutlu Binark, Sevgi Zübeyde Gürbüz, A. Murat Eren (another interview done by Işıl Öz))
Thesis Landscapes from Turkey: Students and Advisors
This section contains numerous thesis published in Turkey. Each example constitutes an unpleasant situation in the academical sense, all of which are the result of different levels of irresponsibility of those who advised these theses. I think it would not be false to argue that the theses I deal with here are unfortunately only the tip of the iceberg.
Advisor: Dr. Güniz Gürüz, METU
The first example is from Middle East Technical University (METU). This thesis pair is the slightest and most negligible one among the ones I will present. The first thesis is as follows:
|University||Orta Doğu Teknik Üniversitesi (Chemical Engineering)|
|Thesis Advisor||Dr. Güniz Gürüz|
|Thesis Committee||Dr. Güniz Gürüz, Dr. Canan Özgen, Recep Parlak, Dr. Suzan Kıncal, Dr. Deniz Üner|
|Thesis||Energy Optimization of the Yankee Hood Dryer, 2000|
The other thesis is as follows:
|University||Orta Doğu Teknik Üniversitesi (Chemical Engineering)|
|Author||Akın Salih Toppare|
|Thesis Advisor||Dr. Güniz Gürüz|
|Thesis Committee||Dr. Güniz Gürüz, Dr. Canan Özgen, Oktay Orhan, Dr. Suzan Kıncal, Dr. Deniz Üner|
|Thesis||The Energy Analysis and the Multivariable Energy Optimization of a Yankee Hood Dryer, 2002|
Both theses, published in 2000, study the most optimal operating conditions of a paper drying device. Both theses introduce the properties of paper, paper manufacturing and a device commonly used in paper manufacturing. The introduction included in the master thesis of Önel published in 2000 and written on the short history and technical properties of the material is substantially quoted (occasionally with the same sentences or with little revisions) in the thesis of Toppare published in 2002:
You can access to an anonymous report analyzing the problems of both theses here: Anonymous evaluation report [PDF].
How much paraphrasing or copy-pasting is OK? Where the line should be drawn? These may seem to be questions with no clear answers. However, what is being tolerated in Turkey falls into the definition of “horribly unethical” in developed countries. Plagarism has very clear cut limits and standards. Drawing these lines in such a way that there is no basis for an ethical hesitance by others is a part of the responsibility of the adviser, which would not only protect students, but may also enforce a better understanding of ethics on the new generation.
Advisor: Dr. Sinan Bilikmen, METU
Another example from METU, one of the most prestigious universities in Turkey. From the thesis of two students graduated by Dr. Bilikmen in the same year, the first one is as follows:
|University||Orta Doğu Teknik Üniversitesi (Physics)|
|Thesis Advisor||Dr. Sinan Bilikmen|
|Thesis Committee||Dr. Sinan Bilikmen, Dr. Gülay Öke, Dr. Cevdet Tezcan, Dr. Serhat Çakır, Akif Esendemir|
|Thesis||“The Non-Uniform Argon DC Glow Discharge System Parameters Measured with Fast Three Couples of Double Probe“, 2006|
And this is the other one:
|University||Orta Doğu Teknik Üniversitesi (Physics)|
|Thesis Advisor||Dr. Sinan Bilikmen|
|Thesis Committee||Dr. Sinan Bilikmen, Dr. Arif Demir, Dr. Gülay Öke, İsmail Rafatov, Dr. Burak Yedierler|
|Thesis||“Measurement of Nonuniform Magnetized Argon Plasma Discharge Parameters“, 2006|
Screen images partially comparing both theses are as follows:
You can access to an anonymous report analyzing the problems of both theses here: Anonymous evaluation report [PDF].
It is very unlikely for Dr. Bilikmen to have missed the remarkable similarity between the theses of two of his students who graduated in the same year. However, he did not see any problem with this. Even the figure which was deformed and became completely unreadable while moving from the initial thesis in the last screen image has drawn attention of neither the advisor nor the committee, and the thesis was put in the archives in this form.
Advisor: Dr. Bekir Karlık, Fatih Üniversitesi
This thesis illustrates a very obvious and equally serious violation of science ethic. Details of the thesis are as follows:
|Author||Ahmet Vehbi Olgaç|
|Thesis Advisor||Dr. Bekir Karlık|
|Thesis Committee||Dr. Bekir Karlık, Dr. Tuğrul Yanık, Dr. Mehmet Şevkli|
|Thesis||“Software Development of Fuzzy Clustering Artificial Neural Networks“, 2009|
People from Turkey will remember Dr. Bekir Karlık from the conference he co-organized in Çanakkale Onsekiz Mart University. Dr. Karlık signed the thesis of his student together with Dr. Yanık and Dr. Şevkli. Some pages of the thesis are as follows:
A great part of this work is the same with the master’s thesis submitted to Houston University by Zhansou Yu (Dr. Yu’s thesis is open-access and you may view it via this address). For the remaining parts of his thesis, Mr. Olgaç gathered content from various other Internet sources and books.
At this point it may be argued that Dr. Karlık -and the other two people in the jury- have suffered from carelessness, and the only one who should be considered guilty is Mr. Olgaç, who is the author of the thesis. However, it is impossible that an advisor who knows the capabilities and competence of his student may not notice the great problem in this thesis. On the other hand, it is also possible to argue that Dr. Karlık is just a bad advisor and did not intentionally tolerate the problems in this thesis, which can be considered slightly more plausible. But it is not the case either. The following example is critical to clarify this point.
Advisor: Dr. Bekir Karlık, Sakarya Üniversitesi
Details of another thesis consulted by Dr. Karlık are as follows:
|Thesis Advisor||Dr. Bekir Karlık|
|Thesis Committee||Dr. Bekir Karlık, Dr. Sabih Aksoy, Dr. M. Ali. Yalçın|
|Thesis||“Yapay Sinir Ağları ile Ünite Planlaması ve Paylaşımı” (“Unit Planning and Sharing with Artificial Neural Networks“), 1997|
When you examine the thesis, you will see that a considerable part of the content signed by Dr. Karlık is also included in the thesis named “Çok Fonksiyonlu Protezler için Yapay Sinir Ağları Kullanılarak Miyoelektrik Kontrol” (Myoelectric Control by Use of Artificial Neural Networks for Multi Function Protheses) submitted by Dr. Karlık himself to Yıldız Technical University in 1993.
Some parts of the thesis are shown below:
I think it is impossible that Dr. Karlık could not remember the parts included his own thesis.
The fact that even the source code given is the same in both leaves no doubt about that this content has been provided by Dr. Karlık to the student.
I think accusing a student about his/her failure in opposing to a method “confirmed” by the advisor would be unjust especially when taking into account the traditions in Turkey. Clearly Dr. Karlık is responsible for what Ahmet Vehbi Olgaç and Selahaddin Kaya did. But does it mean that we have found the guilty?
Answer for this question may be found in the PhD thesis of Dr. Bekir Karlık and his advisor.
Advisor: Dr. Halit Pastacı, Yıldız Teknik Üniversitesi
Details of Bekir Karlık’s thesis:
|University||Yıldız Teknik Üniversitesi|
|Thesis Advisor||Dr. Halit Pastacı|
|Thesis Committee||Dr. Halit Pastacı, (other names in the committee is unknown)|
|Thesis||“Çok Fonksiyonlu Protezler İçin Yapay Sinir Ağları Kullanılarak Miyoelektrik Kontrol” (“Myoelectric Control by Use of Artificial Neural Networks for Multi Function Protheses“), 1993|
There is one more thesis which will complement this example:
|University||İstanbul Teknik Üniversitesi|
|Thesis Advisor||Dr. Mehmet Körürek|
|Thesis Committee||Dr. Mehmet Körürek, Dr. Erdal Panayırcı, Ertuğrul Yazgan|
|Thesis||“Elektromiyografik İşaretlerin Değerlendirilmesi” (“Evaluation of Electromyographic Signs“), 1989|
Following screen images are received from two theses mentioned above:
A critical part of the PhD thesis of Dr. Karlık is the same with the master thesis of Özcan Kuyucu. There is only one reference in Dr. Karlık’s thesis where Özcan Kuyucu’s work is mentioned, and it is in the introduction part, at the end of a sentence which states that there are some other studies held on this topic in Turkey. There is no statement in the thesis regarding that the data used in the study were actually taken from the thesis of Kuyucu. It is impossible that an advisor may not know the potential of a PhD student he/she works with. Therefore, it is unlikely that Dr. Halit Pastacı, who has read and signed this thesis, did not know that the data was not generated by Dr. Karlık. However, having failed in fulfilling the responsibility of leading his student to comprehend a very critical process in his journey, Dr. Halit Pastacı is responsible not only to Dr. Bekir Karlık but also to Selahaddin Kaya and Ahmet Vehbi Olgaç.
I want to emphasize a critical point here: Competence and quality in academia is a tradition.
Advisor: Dr. Ali Okatan, Haliç Üniversitesi
While one of the organizers of the convention held in Çanakkale Onsekiz Mart University was Dr. Bekir Karlık, the other organizer was Dr. Ali Okatan (they even sent me a legal action threat they co-signed after my criticism about the conference they organize appeared in the news in Turkey).
If there really is an academic tradition, it may be argued that Dr. Okatan and Dr. Karlık have similar traditions by considering that they organize conferences together.
Here I will show you how Dr. Okatan treats his students. The following thesis belongs to one of the students Dr. Okatan has graduated:
|Thesis Advisor||Dr. Ali Okatan|
|Thesis Committee||Dr. Ali Okatan, Dr. Yüksel Bal|
|Thesis||“Internet Security Gateways and the Applications“, 2008|
In the figure below parts that are taken from the security notes of the SANS colored in the context of the thesis. Each color shows the beginning and the end of a portion of text copied from a different source. Almost the entire thesis is formed by adding these sources one after another:
The tragicomic aspect of the issue is that every document of the SANS includes the following legal note at the bottom:
“This paper is from the SANS Institute Reading Room site. Reposting is not permitted without express written permission”.
On the other hand, every Turkish citizen has the right to access this thesis since it has submitted to YÖK (Turkish Council of Higher Education). Simply, you may copy parts of this thesis the way you like.
I do not know what kind of motivation led Dr. Okatan to sign this thesis. However, I wouldn’t think that the copyright infringement that takes place here would make the SANS Institute happy. Having signed the thesis in this form, Dr. Okatan does not only cause great damage to his student, but he also exposed Haliç University to a great financial risk, since in theory the university has to keep this thesis in its library, open to the public.
Advisor: Dr. Ali Okatan, Bahçeşehir Üniversitesi
Another thesis consulted by Dr. Okatan, supporting the hypothesis of academic tradition, is as follows:
|Thesis Advisor||Dr. Ali Okatan|
|Thesis Committee||Dr. Ali Okatan, Dr. Adem Karahoca, Dr. Levent Eren|
|Thesis||“Fingerprint Recognition by Using Cosine Transform“, 2004|
A great part of the thesis of Akpolat gathered with exact quotes from the PhD thesis submitted by Salil Prabhakar to the Michigan State University. You can see some parts of it below:
Some other parts of Akpolat’s thesis comes from the homework prepared by a few students on fingerprint recognition in Carnegie Mellon University. Screen capture below indicates in a tragic way why it is impossible for Dr. Okatan to not be aware of what was going on in the thesis of Akpolat:
Having graduated from Bahçeşehir University with the above given master thesis, and made 4 of his 6 publications with Dr. Okatan in the “International Conference on Intelligent Knowledge Systems” (yet another conference organized by Dr. Okatan, in which many miserable articles such as this one published), Çağatay Akpolat is now an instructor in Beykoz Logistics Vocational School of Higher Education and a PhD student in Istanbul University according to his resume I found here (local). Bahçeşehir University has a significant role in this misfortune since the universities are responsible for honesty of people to whom they entrust their students.
When informed about the theses managed by Ali Okatan, who was employed in Karatay University, Department of Computer Engineering, the chancellor of Karatay University, Dr. Ömer Torlak, responds with an e-mail that includes the following remarkable points:
- Even though advisors should also be careful, the actual responsibility for a thesis belongs to the one who writes it.
- An action on Ali Okatan is not going to take place.
To sum up, attitude of Karatay University in such an issue is to blame students and avoid from taking an action on its permanent academic staff.
I will mention the legal and regulatory problems in the following parts of the article. Even though I believe that an audit mechanism under the control of an authority is wrong, and scientific institutions must be truly autonomous, the case of Karatay University is an indicator regarding why the universities are incapable of solving such problems within themselves.
Advisor: Dr. İsmail Naci Cangül, Uludağ Üniversitesi / Balıkesir Üniversitesi
I suspect that it is very often in Turkey where a thesis advisor graduates two PhD students with very similar theses. Here is another example. One of two PhD theses signed by the same advisor is as follows:
|Thesis Advisor||Dr. İsmail Naci Cangül|
|Thesis Committee||Dr. İsmail Naci Cangül, (other members of the committe are not listed in the thesis)|
|Thesis||“Sonlu Cisimler Üzerinde Bachet Eliptik Eğrileri“ (“Backet Elliptic Curves on Finite Objects“), 2009|
The other PhD thesis is as follows:
|Author||Nazlı Yıldız İkikardeş|
|Thesis Advisor||Dr. İsmail Naci Cangül|
|Thesis Committee||Dr. İsmail Naci Cangül, Dr. Osman Bizim, Dr. Ahmet Sinan Çevik, Dr. Setenay Doğan, Dr. Recep Şahin|
|Thesis||“Sonlu Cisimler Üzerinde Frey Eliptik Eğrileri” (“Frey Elliptic Curves on Finite Objects“), 2009|
The single difference in the title of these thesis, remains to be the only difference throughout the rest of them:
The source code given at the end of the theses are also the same:
A piece of code which lost its structure during the copy/paste process:
It is remarkable that such a significant structural problem went unnoticed by the Thesis Committee of 5 people.
In addition, you can also access a report analyzing the problems regarding these two theses: Anonymous evaluation report [PDF].
Being graduated with these theses is a great misfortune for both students.
Advisor: Dr. Erdem Uçar, Trakya Üniversitesi
A similar example from a different advisor and a different university. Here is a master’s thesis:
|Thesis Advisor||Dr. Erdem Uçar|
|Thesis Committee||Dr. Erdem Uçar, Dr. Yılmaz Çan, Dr. Deniz Taşkın|
|Thesis||“Kalp Ritim Bozukluğu Olan Hastaların Tedavi Süreçlerini Desteklemek Amaçlı Makine Öğrenmesine Dayalı Bir Sistemin Geliştirilmesi” (“Development of a System based on Machine Learning with the Aim of Assisting Therapy Processes of Patients with Cardiac Rhythm Disorder“), 2011|
And this is the PhD thesis published after one year:
|Author||Adnan Fatih Kocamaz|
|Thesis Advisor||Dr. Erdem Uçar|
|Thesis Committee||Dr. Erdem Uçar, Dr. Levent Öztürk, Dr. H. Hüseyin Balık, Dr. Yılmaz Kılıçaslan, Dr. Tahir Altınbalık|
|Thesis||“Makine Öğrenmesi Tabanlı Bir Uzman Sistem Tasarımı” (“Design of a Machine Learning-Based Expert System“), 2012|
A significant part of both theses is the same:
Dr. Uçar who consulted both theses is also the only single name included in both committees.
Advisor: Dr. Cevdet Emin Ekinci, Fırat Üniversitesi
I will conclude this pattern by showing two theses signed by Dr. Cevdet Emin Ekinci. I have written about Dr. Ekinci before, after realizing the fact that he published all his ‘international’ publications in Turkish, in a journal, editor of which is also himself. The first thesis managed by Dr. Ekinci is as follows:
|Thesis Advisor||Dr. Cevdet Emin Ekinci|
|Thesis Committee||Dr. Cevdet Emin Ekinci, Dr. Zülfü Çınar Ulucan, Dr. Ömer Keleşoğlu|
|Thesis||“Enjeksiyon Yöntemiyle Üretilen Kristal Yapılı Ferrokrom Cürufu Katkılı Betonların Fiziksel Özelliklerinin Araştırılması” (“Investigating Physical Properties of Crystalline Ferrochromium Dross Compounded Concretes Produced with Injection Method“), 2006|
The other thesis is as follows:
|Thesis Advisor||Dr. Cevdet Emin Ekinci|
|Thesis Committee||Dr. Cevdet Emin Ekinci, Dr. Ragıp İnce, Dr. Ömer Keleşoğlu|
|Thesis||“Enjeksiyon Yöntemiyle Üretilen Granüle Yapılı Ferrokrom Cürufu Katkılı Betonların Fiziksel Özelliklerinin Araştırılması” (“Investigating Physical Properties of Granulated Ferrochromium Dross Compounded Concretes Produced with Injection Method“), 2006|
Both theses were published in the same year. Students begin with acknowledging each other. One thesis deals with crystalline ferrochromium and the other deals with granulated ferrochromium. Some images from the theses which are completely the same except these terms are as follows:
While care is taken to reveal minimal difference at the beginning, even the images become the same towards the end:
I’m afraid most of the advisers and members of thesis committees I mentioned in this section are simply incapable of realizing the damage they are causing to the higher education in Turkey, and there is no serious effort in Turkey to ameliorate this distressful situation.
It is Difficult to Access Theses Published in Turkey
It is very important to make it easy for people to access published theses. However, it is not easy even for scientists to access published theses in Turkey.
YÖK is responsible for the management of the thesis archive, which is supposed to include all theses published in the country. Meanwhile, as usual, universities must keep one copy of each thesis approved by the university in their libraries. But there are many problems with both.
I want to begin with a summary Emrah Göker wrote for this article. He is a researcher frequently writing about the academic problems in Turkey:
“The National Thesis Center” put into service by YÖK a few years ago is a very significant service for the public to be able to access the master and PhD theses published in Turkish universities. However, just a limited number of theses are actually accessible on this web site due to ‘restrictions’. It is not possible to read a thesis if the writer of it restricts the access in the submission form. The second alternative to access a thesis is through the library of the university where the thesis was defended. There are practices differing from one university to another at this point. Access through library is also not possible in some universities if the writer does not permit: such theses are separated from the catalogues of libraries, and stored in inaccessible rooms in graduate schools.
On the paper, there is nothing wrong in making master/PhD theses accessible by the public only with the permission of the author. But explaining the scientific and judiciary reasons is not easy. For instance there are three options in the confirmation form signed by the authors of theses in order to make them available in the UMI/ProQuest Electronic Theses and Dissertations system in the USA: (1) open for worldwide access; (2) open for access only in the University where the thesis was written for the first 5 years (in this case the author must explain the reason of requesting this restriction in the form, the request may be rejected); then the study is opened for worldwide access after 5 years; (3) no access to the thesis for 1 year due to a patent (this requires an additional certification and the access becomes possible in either of the first two options following the end of the first year). Every student must fill this form after they have successfully defended their theses in their institution, and the thesis is signed by the thesis committee.
In the case of Turkey, however, the permission form for public access which YÖK requires authors to fill, permits the closure of a thesis for a period of maximum 3 years without requiring any justification. Morever, theses published before 2006 can only be opened if the writer fills the form and give permission specifically; otherwise YÖK may not take any action. Since the service of ‘photocopying’ is not anymore available, the only chance you have is to go to the university where the thesis was defended, and act like a detective to find the thesis you are looking for, if its author did not send YÖK the form with their own efforts. Let me give two examples: the earliest thesis found in Sociology discipline in the National Thesis Center portal is from 1985. 409 theses were approved until 2005 (inclusive). We can download digital copy of just 41 (10%) of these theses the earliest of which is from 1990. The rule of “restriction for a period of maximum 3 years” has positively affected the access for Sociology since 2006: for thesis published after 2006, only 10 out of 438 theses are not accessible. We can say that digital access to theses has been recovered since 2006 in all disciplines. However, if we consider the big picture, researchers are obliged to seek access to thesis published before 2006 in universities thesis were published, and universities don’t have standard procedures like YÖK.
A Current Example on the Issue of Accessibility
I want to give a real-world example to demonstrate how hard it can be to gain access to a published thesis, and who might be abusing this fact. First, a little bit of back story:
Screen captures below are from an article written by Dr. Halil İbrahim Dursun (Aksaray University) with Dr. Ziya Burhanettin Güvenç (Çankaya University (he was the president of Çankaya University when he took part in this article)) and Dr. Ergün Kasap (Gazi University) in 2009. Their article was retracted from the journal a few months ago. You can see the embarrassing announcement the journal had to make in the last screen image:
This paper is identical to another paper from its abstract to the conclusion. Dr. Halil İbrahim Dursun, who is the first author in this paper, continues his academic life as an Assistant Professor in Aksaray University Department of Physics.
Dr. Dursun earned his PhD degree in Gazi University under the supervision of Dr. Ergün Kasap, who is also one of the authors of this plagiarized article. It is only normal to worry about the integrity of the PhD thesis written by Dr. Dursun under the supervision of Dr. Kasap, considering this plagiarism they commited together.
It is possible to access the thesis information of Dr. Dursun through the thesis archive of YÖK, however, you can’t read it: “this thesis is not accessible, because there is no permission form associated with it”. Simply, Dr. Dursun did not fill the permission form, so no one can read it through the thesis archive.
Arbitrary Regulations in University Libraries Make it Harder to Access Published Theses
So, if you can’t reach it through the thesis archive, what is the other option?
In case of the example above, you can go to Gazi University and obtain a copy of Dr. Dursun’s thesis from the library. Because university libraries must keep one copy available for every thesis that is published in the university.
As a result of my calls in the social media, several volunteers attempted to get a copy of Dr. Dursun’s PhD thesis from the library of Gazi University.
However, all these attempts eventually failed, because the library simply did not give out the thesis.
Of course, this does not imply that there is an ethical problem in the thesis of Dr. Dursun, however, getting such results from these attempts should not surprise anyone. When the public access to questionable documents are regulated by institutes that might get hurt if it turns out that the document is plagiarised, then they don’t want you to have access to it. Briefly, in order not to take a risk of tainting the reputation of the university, they simply don’t let you have it.
Here are two messages from an anonymous academician who had also gone to Gazi University library following my request:
I went to the library of Gazi University and told them that I would like to examine the thesis and take copy of some chapters. After a deep silence, the library officer stated that I could not receive the thesis now. I asked “Why?” He made an explanation that did not satisfy me: “We are now transferring some of the theses in digital media. So we cannot give the thesis you want”. I asked “When will the transfer process be completed and may I receive the thesis after the completion of this process?” In return he asked me “Are you a researcher?”, “Which field are you working?”, etc. I realized that I need to apply another method to receive the thesis.
And this is the final response I received from the same anonymous academician after a few months:
People I asked to mediate (bachelor and master of science students in this university) could not access the thesis either (…) I am sorry I could not help you in this issue.
I hear similar stories from Dr. Tansu Küçüköncü and other anonymous academicians who understand that the investigation of the academia is very important. I gathered the following summary from the problems encountered by people who wanted to obtain some theses that were not avialable through YÖK’s thesis archive. These are the responses they received from theese libraries:
- Gazi University: “We cannot send photocopies of theses. Only some chapters may be copied in the library“. With this, we also see that the response is arbitrary and you are at the mercy of the ‘librarian’.
- Ankara University: “Academic theses are not circulated out of the library. But, taking a photocopy of the whole thesis is permitted in the department building upon permission of the thesis advisor or the person who prepared the thesis“.
- Celal Bayar University: “We do not serve about theses“.
- Istanbul University: “Researchers must declare their advisor and university address while requesting the photocopy of an entire thesis. On the other hand, students must also include their University, Department and ID number in addition to the Advisor. If the person who requests the photocopy of a thesis does not have an advisor, he/she shall write a petition for the İ.U. Department of Library and Documentation regarding the aim of using the thesis. He/she shall deliver the petition enclosing the thesis number and title either in person or through fax. The relevant person shall be notified about the bank account number, photocopy + copyright expense (without adding the delivery fee) to be paid. The notified expense shall be paid in the relevant bank account and the receipt shall be faxed to our library (…)”….. and it goes on and on. They kill you with the ‘bureaucracy’.
- Uludağ University: “Our university gives theses out in the form of photocopy limited to the table of contents, bibliography, abstract and a part of the thesis that is not more than about 20-25 pages”.
- Erciyes University: “We never lend theses or allowed them to be photocopied. You are just permitted to read them in the library“.
- Ege University: “We send the entire thesis if the author has permitted, or just 1/3 if there is no permission“.
- Istanbul Technical University: They send the entire text of theses in photocopy! [An amendment as of 09/10/2012: İTÜ Department of Library and Documentation has sent me a refutation following the publication of this article, in which they stated that what is written here is wrong, and they do not send photocopies: REBUTTAL].
- Balıkesir University: “We do not lend theses. If any thesis is permitted to be copied, it may be accessed as a PDF on our web page“.
- Niğde University: “We do not lend theses. We may send photocopy or PDF version of a thesis to an applicant unless the author of the thesis imposed an embargo saying “nobody would access to my thesis for about 3-5 years”".
- Çankaya University: No response.
- Hacettepe University: No response.
As you can see, university libraries in Turkey can restrict the access to public scientific information arbitrarily.
As a consequence, it is possible and relatively easy to access a thesis submitted to University A while it may be impossible in practice to access a thesis submitted to University B. Such arbitrary restrictions imposed by universities, that are actually being financed from the same tax money, is not acceptable.
I think it very important to implement a standard procedure regarding the accessibility of thesis through university libraries. This way ethically problematic theses can be revealed quickly, and people who signed these theses may be prevented from advising more PhD students who may inherit their academical incompetencies.
There is a Great Number of Theses Without Permission for Public Access in YÖK Archives
As emphasized in the abstract Emrah Göker wrote, it is relatively possible to access theses that were published after 2006 through YÖK’s Thesis Archive. However, the number of theses published before 2006 is considerably high and almost none of them are accessible.
In theory, YÖK Thesis Archive lists to 312,368 titles overall. But this number is rendered insignificant with the fact that 163,284 of them (52.27%) are closed for access. The following table is prepared with data obtained from YÖK Thesis Archive, and shows the first 30 universities where the ratio of open vs. closed theses is lowest.
|Orta Doğu Teknik Univ.||5,785||10,391||16,176||%64.24|
|İstanbul Teknik Univ.||5,416||9,072||14,488||%62.62|
|Ondokuz Mayıs Univ.||1,961||2,178||4,139||%52.63|
|Dokuz Eylül Univ.||5,743||6,191||11,934||%51.88|
|Yıldız Teknik Univ.||3,300||3,467||6,767||%51.24|
|Mimar Sinan Güzel Sanatlar Univ.||1,107||1,100||2,207||%49.86|
|Karadeniz Teknik Univ.||2,337||2,319||4,656||%49.81|
|Yüzüncü Yıl Univ.||1,584||1,516||3,100||%48.91|
|Gebze Institute of Technology||1,072||871||1,943||%44.85|
It is not surprising that theses prepared in old and established universities are mostly closed, because they have many more theses written before 2006.
The present state of the Thesis Archive is not acceptable for a healthy, self-correcting, self-controlling academia.
YÖK must set up regulations for immediate release of all submitted theses. Academics must do their part by pushing YÖK to put this item into its agenda.
No Clear Reaction to Scientific Theft in Turkey
The inaccessibility of theses is a significant problem. It is essential to solve this problem collectively. However, even in cases where there is a substantial evidence regarding the occurrence of plagiarism, actions taken by the authorities are far from being satisfactory compared to the equivalents in developed countries. As a result, efforts fail, and violations remain covered up in universities that seem to be above the law.
Recent Examples from Europe
News on plagiarism occupying the public agenda in Europe in the recent months, and the response from the society against these news are inspirational. In contrast, Turkey has failed in progressing to criminalize plagiarism, and even YÖK is now merely retrogressing with the new regulations.
I want to remind some of the events and the aftermath:
- Karl-Theodor zu Guttenberg, Defense Minister of Germany: It was revealed in February 2011 that Guttenberg plagiarized in the PhD thesis on Law he wrote in 2007. Guttenberg announced immediately after the emergence of the claims that he ceased to use the title of doctorate temporarily. After a short time, he asked the University that granted him the degree of doctorate to cancel his degree due to ‘the serious faults he made’. His degree of doctorate was cancelled by the Bayreuth University council after two days. The Prime Minister of Germany, Angela Merkel, presumed to support Guttenberg, who is a member of Merkel’s cabinet. The society answered back Merkel with an open letter signed by 51.000 PhD students and researchers. Guttenberg resigned in March 2011.
- Pál Schmitt, President of the Republic of Hungary: Having sustained his office in presidency of the republic since May 2010, Schmitt was accused in January 2012 with the claim that he had plagiarized in his PhD thesis published in 1992 from another PhD thesis published in 1987. The Presidential office initially responded by asserting that both authors had studied from the same sources and this is the reason of similarities. Semmelweis University where the PhD thesis was submitted started to search the issue and reached the conclusion that Schmitt prepared a considerable part of his thesis by translating from the other thesis. The University repealed the title from Schmittin in March 2012. Schmitt resigned in April 2012.
- Ioan Mang, Minister of Education of Romania: As a member of the Social Democratic Party in Romania, Mang was accused with the claim that he had plagiarized in his several academic publications in May 2012. Mang argued that these claims are underlined by political motivations. The ademic community of Romania gave a rapid and great response upon the claims emerged. Mang resigned after 5 days from the emergence of the claims.
- Silvana Koch-Mehrin, German Politician and the Member of European Parliament: As a result of the searches of volunteers who follow hard upon the plagiarisms in Germany after the Guttenberg case, it was claimed that Koch-Merhrin plagiarized in one fourth of his thesis written in 2000. These claims put Heidelberg University in action in April 2011. The research committee established by the University reached some results which confirm the claims. PhD degree of Koch-Merhrin was repealed in May 2011. Koch-Merhrin resigned from the membership of European Parliament and from his party.
The process of cleaning plagiarism for the stability and the honor of the academy is still continued in Europe.
Those who have followed-up the responses of academicians and politicians accused of scientific plagiarism or violation of scientific ethics and the general silence of the society, particularly Turkish academy, in this issue for years will undoubtedly witness the days in which Europe will start to obtain the outcomes of today’s attempts with a bittersweet happiness.
Lack of Motivation Arising from Administrative Problems
There are several reasons why an academic cleanup comparable to European examples is not being done in Turkey.
One of them is the difficulty in accessing dissertations mentioned in the previous chapter.
Another reason, equally important, is the lack of motivation that is required to fight this war. One important reason for the lack of motivation is that, thanks to lax regulations, even people whose plagiarism is documented beyond proof stay in their posts.
Three names that are mentioned in an example above, Dr. Halil İbrahim Dursun, Dr. Ziya Güvenç, and Dr. Ergün Kasap, are still serving in their departments. An anonymous member of administrative staff in Aksaray University, where Dr. Halil İbrahim Dursun currently serves as assistant professor, has shared their opinion in an online discussion about what will not happen to Dr. Dursun.
I believe this depressing information will shed light on the reasons for the lack of motivation.
It came up in the previous rector’s term. We quickly investigated, and then realized that it indeed bears too much similarity with a cited source, as indicated in the news story.
Our regulation regarding plagiarism defines it as “To pass someone else’s scientific publication or work, wholly or in part, without citing, as his own”. It clearly states “without citing”. In the present example, there are too much text that is taken, but with citation. Whatever our ethical principles say, we cannot disregard the [legal] side of this. In short, we need to realize that our legal system has loopholes.
In the present case, the punishment will be light, possibly a warning, a reprimand, or a pay cut. YÖK will not agree to a stronger punishment, because in the legal process the professor will win, for the reasons above.
I wanted to provide this statement as completely my personal opinion. Naturally, it is the rector who is entitled to make an official declaration in the name of Aksaray University.
When the administrative green light about ethical violations is combined with insufficient legal regulations, it becomes impossible to see any professors resigning their posts, in contrast to European countries. Furthermore, we shoot ourselves in the foot by letting these people to educate students and allowing them to administer academic institutions.
Two Examples About Academic Apathy
Surely everybody reading this review will remember the scandal of 2005-2006 where 65 papers originating from Turkey are found to be plagiarized and helped Turkey to make a name for itself in scientific community. What do you think has happened to the protagonists of this historic scandal?
I compiled a list about their occupations at the time of this writing:
- Mustafa Saltı, whose 40 papers have been retracted, was a doctoral student then. Today he is a research assistant in Dicle University Physics Department.
- Dr. Oktay Aydoğdu, whose 29 papers have been retracted, was a doctoral student then. As of now, he is a reserch assistant in Mersin University Physics Department.
- Dr. Sezgin Aygün, whose 15 papers have been retracted, was another doctoral student at that time. Now he works as an assistant professor in Çanakkale Onsekiz Mart University Physics Department.
- Murat Korunur, another doctoral student whose 14 papers have been retracted, currently works as a research asistant in Dicle University Physics Department .
- Dr. Ali Havare was an assistant professor when his 13 papers have been retracted. He is currently an associate professor in Mersin University Physics Department (he also served as a thesis supervisor for Mustafa Saltı, Dr. Oktay Aydoğdu and Taylan Yetkin, all actors in this same plagiarism scandal.)
- Dr. İsmail Tarhan was an associate professor when his 13 papers were retracted. He is currently a full professor at Çanakkale Onsekiz Mart University Physics Department. In 2011 he was serving as director of Science & Engineering Graduate School at ÇOMU, and as rector’s consultant.
- Melis Aygün (now Melis Ulu) was a doctoral student when her 10 papers were retracted. She is now a research assistant at Çanakkale Onsekiz Mart University Physics Department.
- Dr. Hüsnü Baysal, whose 7 papers were retracted, was an associate professor then. He is currently a full professor at Çanakkale Onsekiz Mart University, Department of Secondary-level Science and Mathematics Teaching.
- Dr. İrfan Açıkgöz, whose 5 papers have been retracted, is still a professor at Dicle University Physics Department (he also supervised the thesis of Murat Korunur, who is mentioned above).
- Dr. İhsan Yılmaz, whose 4 papers have been retracted, is still a professor at Çanakkale Onsekiz Mart Unversity, Department of Space Sciences and Techonologies. In 2011, he was also serving as director of Revenues Office and as Vice Rector at ÇOMU.
- Dr. Figen Banbay, whose 3 papers are retracted, is still an assistant professor at Dicle University Physics Department.
- Dr. Nurettin Pirinççioğlu was a doctoral student when his 3 papers were retracted. He is currently an assistant professor at Dicle University Physics Department.
- Can Aktaş, then a doctoral student, had 1 paper retracted. Today he is an assistant professor at Çanakkale Onsekiz Mart University Mathematics Department.
- Taylan Yetkin was a lecturer at Mersin University Physics Department when his 3 papers were retraced. Today – as far as I can see – he is not affiliated with academia. In this list, he is the only one who is not in academia any more.
I have no intention of offending anyone in this list, some of whom I personally know (it’s not hard to imagine how deeply they are affected by this incident), but I think this picture provides a very important indicator.
Back then, when this incident caused a large-scale discussion in Turkey, Dr. İhsan Yılmaz wrote a letter to Nature where he admits to have copied only the introductions of some papers, “in order to express the problem with better English”, and he claims that everyone is doing this and it cannot be called plagiarism. Dr. Mehmet Özgür Öktel, a physicist at Bilkent University, responded that their plagiarism is proven and not everyone is doing it . Debates raged on, but involved persons were acquitted by their universities.
Dr. Bengü Sezen provides a second example.
After her graduation from METU, Bengü Sezen started her doctoral studies at Columbia University. Toward the end of 2010, it became clear that Dr. Sezen was involved in serious academic fraud during her years at Columbia. According to reports, for ten years, Dr. Sezen tampered with experimental results in an effort to make them seem more interesting, and she created imaginary people and organizations in order to make other scientists believe the verity and reproducability of her impressive and controversial results.
Columbia University published a 167-page report on the subject, and Sezen’s degree was revoked. However, she had left Columbia before that and enrolled in a new doctoral program at Heidelberg University. On the other hand, it is claimed that even though Sezen got her degree in 2009, her dissertation is not found in German National Library archives, where every dissertation published in Germany must be listed (I thank felixis, who is an anonymous academician, for bringing this to my attention).
I would also like to note that I sent two e-mails to Dr. Sezen’s Ph.D adviser, Elmar Schiebel (on 10 Sep 2012 and 17 Sep 2012), to ask his opinion about why Dr. Sezen’s dissertation is not in German National Library archives. However, got no responses back.
Several embarrassing articles about Sezen and the Columbia University report can be accessed with the links below.
- Reports Detail A Massive Case Of Fraud (William G. Schulz)
- A Puzzle Named Bengü Sezen (William G. Schulz)
- The Sezen Files – Part I: New Documents (Paul Bracher)
- The Sezen Files – Part II: Unraveling the Fabrication (Paul Bracher)
- The Sezen Files – Part III: And What of Sames? (Paul Bracher)
The first article ends with the following observation:
(…) Her legacy of betrayal, observers say, remains one of the worst cases of scientific fraud ever to happen in the chemistry community.
Dr. Sezen goes on with her academic life in Turkey.
According to the web site of Gebze Institute of Advanced Technology, as of January 2012, she is appointed (screenshot) as assistant professor in the Bioengineering Department (screenshot) in the Institute’s Faculty of Engineering.
People make mistakes. I don’t think we should expect their professional lives should end because of these mistakes. Those who write on the subject keep repeating that Dr. Sezen’s fraud can be possible only with a deep knowledge and understanding of chemistry. So, I have no doubt that Dr. Sezen has the potential of becoming a successful scientist. However, I don’t think she can find a place in the scientific community in any developed country, given her 10-year history of fraud, which begs the question: Why is it acceptable in Turkey?
Scientists are familiar with similar cases. In 2011 it was revealed that the Dutch psychologist Diederik Alexander Stapel, who was considered extremely bright and trustworthy by those who know him, has fabricated experimental results for many years. Stapel returned his doctoral degree, and released the following apology:
I failed as a scientist. I used fabricated data in my researches. Not once but for several times. Not for a short while, but over a long period. I am aware that I shocked my colleagues due to my misconduct. I put my field of study, social psychology in trouble. I am in shame and feel sorry
I declare my sincere apologies to my colleagues, PhD students and the entire academic community. I am aware of the suffering and the grief I have caused
I could not brave the pressure to be successful, to publish and to be better in time. I wanted too much too soon. I have gone astray in a field where there is a little control and people usually work alone.
Even though he admitted his mistake, thereby showing that he still possesses some of the qualities of being a scientist, Stapel is not working anywhere in academia now.
Similarly, German physicist Jan Hendrik Schön left science after allegations of misconduct in 2002. Likewise, South Korean biologist Hwang Woo-suk, whose misconduct was caught in 2006, is not employed anywhere in academia (further, he was sentenced to jail for violating the bioethics law of South Korea).
When investigating such affairs, I believe we need to maintain a healthy skepticism.
It should be considered that these names could be victims of injustice, and things may not be black and white. Further, before pulling our swords of criticism, we should consider the trauma and stress experiences by these people.
On the other hand, independent of individuals and excuses, we see that almost all the names in the earlier examples moved on with their careers unscathed; even got promoted. This is a horrible indicator of the ineffectiveness of regulations that are supposed to protect Turkish academia, and of the apathy of Turkish academics. We need to perceive that this apathy creates the feeling that “Turkish academia has rewarded even those who were involved in the biggest scandals, what is the worst than can happen if I’m caught?”, this feeling can diffuse across the entire community, and in such a community the theft of Dr. Halil İbrahim Dursun may seem like an innocent detail.
Evidently we must make some advances toward preventing that.
Regulations that Retrograte, and the Ignored Report of the “State Auditing Board”
Even though it is obviously imperative that we need to improve in this area, small and scattered legal changes are far from being an advancement. I would like to quote a small note written for this article by an anonymous professor who teaches in a Turkish university:
About the new student discipline regulation, YÖK director Çetinsaya says: “Plagiarism in academic work such as thesis, class paper, or seminars now requires disciplinary action” (Official Gazette, Volume 28388).
Students who have committed ‘plagiarism’, defined as stealing from another study or book for their seminars, theses or publications, will be punished with suspension for ‘one academic semester’.
Defining plagiarism as an academic crime for students is a positive development. However, the punishment is insufficient and not in any way an effective deterrent. The right to investigate belongs to faculty deans and institute directors, who may cover up the allegations, as in many similar cases.
And what about the professors of these students? Will there be strong precautions regarding them, too? Will they be subject to disciplinary action, too?
How about reporting ethics violations to an independent body, above universities and YÖK?
However, such a body is unlikely to be accepted and established, seeing that even the report of the State Auditing Board of the Presidency (DDK, “Devlet Denetleme Kurulu” in Turkish) titled “Oversight and Supervision in Higher Education – Legal Framework and Practices” is largely disregarded.
It is obvious that these practices will not help us to get somewhere better.
The conclusion of the State Auditing Board Report on the problems of higher education is pretty striking. You can read a summary here (Turkish), but I’d like to quote the English translation of the conclusions here, as I find it extremely important:
Because of insufficient legal infrastructure regarding investigations involving auditing and penalties, and because of the actions of The Council of Higher Education (YÖK), YÖK directors, and Higher Education Auditing Board, proper supervision and auditing of the universities by the government can no longer be carried out as a public service.
In other words, present structural problems of Higher Education Auditing Board and its deficiencies/errors in auditing, as well as the illegal actions of YÖK administration and rectors regarding complaints and reports, crippled the auditing functions required by the Constitution.
As a result, perceptions of directors of higher education institutions regarding accountability has changed, so the field of higher education has become an “environment” where corruption is not prevented, even increased. Therefore the existing “auditing gap” seems to have turned into a chronic structure that feeds itself.
An incredible report.
I think it’s very clear: Turkey urgently needs a large-scale academic revolution.
Turkey’s bad academia is self-perpetuating.
People who have committed ethical violations in their dissertations and publications are allowed to become thesis supervisors. Students who are misguided by these create dissertations that equally violate ethics, publish insignificant or duplicated papers, and some of them become the new academic generation, in turn completing the cycle.
One of the major problems that perpetuates this cycle is the difficulty of access to dissertations. University libraries limit access with arbitrary reasons, and improvements in YÖK Thesis Archive are far from solving the problem in practice.
Even when a dissertation is accessed and plagiarism is seen, penalties are far from being deterrent, due to legal and executive roadblocks.
While advanced societies take science theft very seriously, actors of science theft in Turkey silently go on with their duties, thus deleteriously undermining the credibility of the field.
Even though today’s scientists in Turkey are not proactive, and they are mostly mute unless they have to defend themselves, I believe that self-criticism will become a way to reveal and eventually eradicate academical problems in Turkey in the future.
These are my final words for the academics, students, and all activists of Turkey:
You must carefully analyze your responsibilities in breaking this vicious cycle of incompetency in science.
Academia in Turkey needs a reform, and there is something to do for everyone who is willing to be a part of this struggle; including raising awareness in the society about the dangers of plagiarism, loudly expressing the legal and executive problems, exposing and publicly condemning cases of scientific theft, helping these criticisms be heard by the rest of the society and by the people whose are paid to protect public interest with their administrative power, such as the members of the senate.
Academia in Turkey is crippled by a common irresponsibility, and a permanent reform is only possible by everyone accepting their responsibility.
Eren is a post-doctoral scientist studying microbial ecology in the United States of America (a.murat.eren / gmail.com).