TURKISH JOURNAL OF ONCOLOGY 2021 , Vol 36 , Num 2
The Evaluation of the Validity and Reliability of the Breast Cancer Awareness Scale (for Turkish Women)
Kamile ALTUNTUĞ1,Vesile KOÇAK1,Emel EGE1
1Department of Obstetrics and Gynecology Nursing, Necmettin Erbakan University Faculty of Nursing, Konya-Turkey DOI : 10.5505/tjo.2021.2708

Summary

OBJECTIVE
The aim of this study is to evaluate the validity and reliability of the Breast Cancer Awareness Scale (BCAS) in Turkish.

METHODS
This study is a methodological study. The validity and reliability of the B-CAS were evaluated.

RESULTS
The Kaiser-Meyer-Olkin value was found to be 0.76, while the Bartlett"s test results were found to be X2=6411.088, and both test results were found to be at p=0.00 significance level. As a result of the analyses, a five-factor structure that explains 57.85% of the variance and has factor loadings ranging from 0.63 to 0.82 was obtained. It was found that the sub-dimensions of the scale have a good internal consistency. The Cronbach Alpha internal consistency coefficients of the sub-dimensions of the scale were found as 0.87 for Knowledge of Risk Factors, 0.88 for Knowledge of Signs and Symptoms, 0.81 for Attitude to Breast Cancer Prevention, 0.78 for Barriers of Breast Screening, and 0.61 for Health Behavior related to Breast Cancer Awareness.

CONCLUSION
The analyses revealed that the B-CAS is a valid and reliable tool. The Turkish version of the 29 item BCAS can be used as a valid and reliable measurement tool.

Introduction

Cancer is a common and important health problem worldwide. As stated by the World Health Organization (WHO), 9.6 million people died due to cancer only in 2018.[1] According to the data published by the WHO in 2018, 210 thousand 537 new cases of cancer were reported in Turkey.[1] According to the Turkish Ministry of Health data, an average of 163 thousand cancer cases is reported every year in Turkey. In 2015, 177 out of every 100 thousand women and 247 out of every 100 thousand men were diagnosed with cancer.[2] According to the World Cancer Research Fund (WCRF) data, the most common cancer types in the world in 2018 were lung cancer, breast cancer, and intestinal cancer, respectively.[3] Breast cancer is the second most common cancer in women and the second most common cancer worldwide. There were more than 2 million new cases in 2018. The WCRF reported that breast cancer is most common in Australia (468/100 thousand), followed by New Zealand (373) and Ireland (438).[3] In Turkey, one in every four female cancer patients has breast cancer.[2] When the age distribution of breast cancer in Turkey is analyzed, it is seen that the age group in which breast cancer is the most common is the 25-49 age group with 34.2%, compared to other age groups.[4]

Although breast cancer mortality rates decrease in many high-income countries, incidence and mortality rates increase especially in fast-developing countries. [5] This increase is associated with changes in lifestyle, changes in reproductive functions, nutritional habits, population growth, aging, and the increase in screening methods.[6,7] The WHO stated that cancer cases can be prevented by controlling environmental conditions and through age changes. The WHO emphasizes that the most cost-effective long-term strategy for cancer control is cancer prevention and at least one-third of all cancer cases are preventable.[8] It is estimated that 30-50% of cancer cases can be prevented by avoiding risk factors and implementing evidence-based prevention strategies.[1] Ensuring public awareness about cancer, developing community awareness and cancer screening are among the most effective methods in the fight against cancer. In Turkey, cancer screening is carried out as recommended by the WHO. In 2014, the standards to be followed during the community-based breast cancer screening program studies were redefined by the Public Health Institution of the Ministry of Health and published under the title "The National Standards of the Breast Cancer Screening Program." Accordingly, it has been acknowledged that screening mammography be carried out among all the women aged 40-69 years every 2 years. In an effective screening program, it is aimed that more than 70% of the target population participate in the screening. To create social awareness, it has been recommended to give consultancy service to every woman above 20 years of age to perform breast examination on their own.[9] Within the scope of the breast cancer screening program, consultancy service is provided to women so that they do Breast Self-Examination (BSE) monthly, women between the ages of 40 and 69 years undergo clinical breast examination once a year and mammography once in 2 years.[10] In 2018, 34.8% of the women in Turkey had mammography. This rate is quite low compared to many European countries.[1] To implement screening programs, raising awareness od breast cancer among women is essential.

Breast cancer incidence in Turkey was reported to be 24/100.000 in 1993, while it increased to 50/100.000 in 2017, indicating a more than double increase.[7] Delayed breast cancer diagnosis in developing countries may be related to poor breast cancer awareness and barriers to access to health-care services.[11-14] Breast cancer awareness is not strong. Particularly low levels of education in women, socio-cultural characteristics and belief systems, including fatalism and moral reasons, may affect awareness.[15-18] Low health literacy levels pose an important obstacle that affects screening attitudes for breast cancer knowledge and the source of information among women.[19] Despite increasing screening centers and free mammography (40-69 years) and mobile screening systems in recent years, a significant number of patients in Turkey are diagnosed at an advanced stage due to the lack of breast cancer awareness.[7]

Increasing breast cancer awareness is considered as the first step in both primary and secondary prevention and commonly in the fight against breast cancer. It is important to understand and improve the level of breast cancer awareness.[20] The first step of breast cancer awareness is the availability of valid and reliable tools that measure this construct and the related factors. Appropriate interventions can be designed and evaluated to increase breast cancer awareness even with only such tools. Various tools have been proposed to evaluate breast cancer awareness.[21-25] As a result of the literature review, no accepted measurement toolhas been found to evaluate breast cancer awareness in Turkey. Thus, this study aimed to evaluate the validity and reliability of the Breast Cancer Awareness Scale (B-CAS) to determine the level of awareness about breast cancer.

Methods

Research Type
The B-CAS was developed by Rakkapao et al. (2016) for the early diagnosis and prevention of breast cancer. It is easy to use and was designed as a self-report tool that allows researchers and practitioners to better understand women"s awareness level of breast cancer. The original scale consists of five factors and 35 items. The factors are Knowledge of Risk Factors, Knowledge of Signs and Symptoms, Attitude to Breast Cancer Prevention, Barriers of Breast Screening, and Health Behavior related to Breast Cancer Awareness. This study is a methodological study. Turkish validity and reliability of the B-CAS were tested.

The Research Context and its Characteristics
The study was carried out in a Family Health Center in Konya. In the Family Healthcare system, women of all ages are integrated into the system. Women receive counseling through invitation, and their health check is done.

Research Population and Sample Size
All the women in the 20-69 age group registered in the family health system constituted the research population. In Turkey, breast cancer is among the most common cancer types in women. After 20 years of age, women receive counseling for BSE and women between the ages of 40 and 69 are scanned for breast cancer through the invitation method every 2 years. [26] Based on this data, all women between the ages of 20 and 69 registered in the family health system constituted the research population. In validity and reliability studies, it is suggested that sample size should be 5-10 times of the number of items so that analyses could be carried out.[27-29] Since there are 35 items in the BCAS, it was determined that there should be at least 350 people, which is 10 times higher than the number of items.

Research Inclusion Criteria
The following criteria were included in the study:

• Having no diagnosed disease related to breast
• Being literate in Turkish
• Having no psychiatric problems (to be determined based on self-report).

Research Exclusion Criteria
The following criteria were excluded from the study:

• Having been diagnosed with breast cancer
• Being illiterate.

Data Collection Technique and Tools
The Personal Information Form and the B-CAS were used to collect data. The Personal Information Form consists of 12 questions including the socio-demographic characteristics of women. The questions are related to participants" age, educational status, employment status, monthly income, marital status, number of children, having experienced menopause or not, first menstrual age, being a smoker or not, cancer history in the family, etc.

The B-CAS was developed by Rakkapao et al. (2016) for the early diagnosis and prevention of breast cancer. The scale consists of five factors and 35 items. The five factors are Knowledge of Risk Factors, Knowledge of Signs and Symptoms, Attitude to Breast Cancer Prevention, Barriers of Breast Screening, and Health Behavior related to Breast Cancer Awareness. In the scoring of the scale, Knowledge of Risk Factors and Knowledge of Signs and Symptoms factors are marked as "Yes," "I do not know," and "No" In scoring, 2 points are received for the correct answer, 1 point is received for the answer "I do not know," and 0 point is received for the wrong answer. The factors of Attitude to Breast Cancer Prevention, Barriers of Breast Screening, and Health Behavior related to Breast Cancer Awareness are scored on a five-point Likert scale as 1 point for "strongly disagree" and 5 points for "strongly agree."

The study is composed of two stages. At the first stage, the scale was translated from English to Turkish to ensure language validity, and translation validity was tested. For this purpose, first of all, permission was obtained from the developers of the scale to carry out the Turkish validity and reliability study. The translation of the scale was made by experts who know both languages and know how to use the scale. The scale was first translated into Turkish by two experts who are experts in their fields and proficient in English. It was checked whether these two translations matched.

Content validity is concerned with examining to what extent the scale measures the construct that is intended to be measured. If the developed test includes all the important subtopics of the subjects that are examined, it is accepted that the test has content validity. It is recommended to seek expert opinion for this purpose. The expert group should consist of at least three and at most 20 people.[29] Experts are asked to evaluate whether the items produced represent the construct to be measured, whether the items are sufficiently simple and clearly expressed, and whether they are expressed appropriately for the target audience. Content Validity index (CVI) is used to prove content validity with numerical values. For this evaluation, experts are asked to score each scale item between 1 and 3.

1 point: Not appropriate
2 points: The item needs to be revised
3 points: Appropriate.

Evaluation: (Number of experts who say the item is necessary/[Total number of experts/2]-1).

If the scale has two or more dimensions, CVI must be calculated separately for each dimension. For the scale to have content validity, the CVI score must be 0.80 and above.[29]

Construct Validity
Construct validity is concerned with to what extent abstract concepts, which cannot be directly observed, are measured correctly, to what extent the goal is achieved, and how accurately it has been measured. There are four methods to evaluate construct validity: Factor analysis, the comparison of opposite or known groups, hypothesis testing, and multivariate multi-method matrix method. In this study, the factor analysis method was used. Factor analysis is a statistical method that allows to combine a large number of related variables and to express them with fewer new variables.[30,31] Factor analysis consists of two parts: Exploratory and confirmatory factor analysis.[29,32] In our study, exploratory factor analysis was used.[29]

Exploratory factor analysis is a method used to reveal under how many sub-headings the items in the scale will be grouped and how these groups relate to each other.[31] The sample size should be sufficient for exploratory factor analysis, and there are various opinions on this. While some say the sample size should be 5 or 10 times the number of items in the scale, others say sample size should be three to five hundred people.[31] The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was used to determine that the sample size was adequate. As the value obtained at the end of this measure approaches 1, sampling adequacy increases, whereas if the value moves away from 1, sampling adequacy decreases. According to some researchers, sample size of at least 0.60 is adequate, while for others, it should be 0.80. Sample size between 0.80 and 0.90 is generally considered to be very good, while sample size over 0.90 is considered to be excellent. For the exploratory factor analysis, first, KMO, which tests the suitability of the data set for factor analysis, and the Bartlett Test, which examines the correlation between the variables, were applied.[31] A valid test should also be reliable. Thus, the internal consistency reliability test was performed to determine whether all aspects of the scale are capable of measuring. For this purpose, Cronbach's Alpha Reliability coefficient and Total Item Score Scale reliability Kappa fit coefficient were examined.

Ethical Considerations
The participants were informed both verbally and in writing about the study and were included after providing their informed consent in compliance with the Helsinki Declaration. Ethics committee approval was obtained from Necmettin Erbakan University, Meram Medical Faculty Hospital, Clinical and Non- Pharmaceutical Research Ethics Committee. For validity and reliability scale necessary permits were obtained from the owner Dr. Cameron Hurst. Necessary permissions were also obtained from the administrators of the Family Health Center, where the study was conducted. In addition, after giving detailed information about the research to women who would participate in the study, their written and verbal consent was obtained.

Limitations
The research is limited to the answers given only by the participants.

Statistical Analysis of the Data
The SPSS and AMOS software programs were used for the statistical analysis of the data.

Results

In this study, the psychometric properties of the B-CAS were tested for Turkish sample. The original scale has five dimensions which are Knowledge of Risk Factors, Knowledge of Signs and Symptoms, Attitude to Breast Cancer Prevention, Barriers of Breast Screening, and Health Behavior related to Breast Cancer Awareness. The first two dimensions include questions with "Yes-I do not know-No" options, while the other three dimensions include questions that are evaluated on a five-point Likert scale from "1 - I strongly disagree" to "5 - I strongly agree."

Participant Characteristics
The average age of the participants was found to be 34.68±10.90 (Table 1).

Table 1: Demographic characteristics of the participants

Language and Content Validity Analysis
To ensure language equivalence, the original scale was translated from English to Turkish by a language expert who knew and understood both languages well, and the translated statements were back- translated to English by another language expert. Three academicians who are experts in both languages evaluated whether there was any meaning loss in the translated version compared to the original scale. During this evaluation, the items in the scale were compared and language validity was assessed by checking the integrity of meaning.

Validity is an important criterion for the credibility and generalization of the study.[33,34] In validity studies, it is important that the correlation between the scale to be developed and what is intended to be measured is consistent. This is called content validity, through which how much each item serves the purpose is determined. With this method, the items that best inrepresent the research topic can be determined from among the items in the scale and irrelevant items can be excluded.[33,35-38] Different techniques have been developed to test content validity.[39-44] One of the most widely used techniques is the technique developed by Lawshe (1975).[35,45] This technique is both simple and useful.[46]

In this study, first, the expert group was created, the initial scale was prepared, and expert opinions were obtained. Then, the content validity ratios (CVR) were calculated with the data obtained. The quality and number of experts (between 5 and 40) are important to assess content validity.[35,38,40,45] In this study, opinions of 11 experts were received. The expert form included the 35 items to determine content validity. The experts were asked to mark one of the alternatives (1 - The item is not appropriate, thus it needs to be removed from the scale; 2 - The item is appropriate, but needs to be revised; and 3 - The item is appropriate) to evaluate the items in the expert form. In Lawshe (1975) technique, expert opinions for each item are rated as "the item measures the target construct," "the item is related to the construct, but not essential" and "the item does not measure the target construct." In this study, the items were scored as "Appropriate" 3 points, "Appropriate but should revised" 2 points, and "Not necessary" 1 point. In addition to the rating in the Lawshe (1975) technique, if the experts have selected the option, the researchers also asked the following questions to the experts: "If your response is "Appropriate but should revised," what is your suggestion?" and "If your response is "Not necessary," why do you think the item is not essential?"

In this study, it was decided to take the values of CVR proposed by Ayre and Scally (2014) as the basis. The CVR is calculated by dividing the number of experts who classify an item as "essential" (E) by the total number of experts divided by 2 (N/2) and subtracting 1 from the resulting number ([E/[N/2]]-1). This calculation is made for each item and it is evaluated whether the item is suitable considering the value in the table calculated according to the number of experts.[46] Ayre and Scally (2014) reported that the critical value of CVR should be 0.636 for 11 experts. CVI is the average of the CVR value of the items left in the pool. In addition, some grammatical and spelling errors in the scale were corrected. Most of the items were scored as "appropriate" by the experts and the CVI was found to be 0.93 (Table 2). In line with the suggestions of the experts, the integrity of the expressions was ensured in the scale items. The content validity of the scale was completed by piloting the final version of the scale in a group of 15 people in terms of readability and understandability in Turkish.

Table 2: Expert opinion content validity index calculation

Construct Validity
Before the scale"s construct validity, KMO value and Bartlett"s test results were examined. KMO was determined as 0.76, and Bartlett"s test result was X2=6411.088. Both test results were found at p=0.00 significance level. Within the scope of this study, the factor structure of the scale was analyzed using the principal components analysis and the KMO value was found to be 0.76. As a result of the analysis, 11 factors with eigenvalues >1 and explaining 71.73% of the variance were obtained. However, since the items in these factors did not constitute a meaningful structure, the scale was forced to a 5-factor structure and the analyses were repeated, based on the factor analysis results of the original scale and the analysis proposed in the scree-plot graph. After the Varimax axis rotation, a 5-factor structure, which explained 50.33% of the variance and had factor loadings ranging from 0.30 to 0.81, was obtained. When the factors under which the items were loaded were analyzed, it was seen that the item "The history of breast cancer in the family" was under the Knowledge of Risk Factors Factor in the Original Scale; however, in this study, it was loaded on the factors of both Knowledge of Risk Factors and Knowledge of Signs and Symptoms. It was decided to exclude the item from the analyses since the difference between the loadings on two factors was <0.10. Furthermore, the item "How many days a week do you eat fresh vegetables?" under the Health Behavior related to Breast Cancer Awareness dimension was found to have a factor loading below 0.30, and the items "How many days a week do you exercise or do sports?" "Have you heard about the breast screening policy of the health-care personnel in your neighborhood?" "How often do you have clinical breast screening?" and "How often do you have mammography?" were found to have corrected total item correlation values below 0.30; thus, all these items were removed from the analyses. After the six items were removed, the analyses were repeated.

As a result of repeated analyses, a five-factor structure, which explained 57.85% of the variance and had factor loadings ranging from 0.63 to 0.82, was obtained. All the items in the structure obtained were found to be compatible with the original scale. In addition, it was observed in the research sample that the sub-dimensions of the scale had good internal consistency. The total Cronbach Alpha internal consistency coefficient of the scale was calculated as 0.71. Cronbach Alpha internal consistency coefficients of the sub-dimensions of the scale were calculated as 0.87 for the Knowledge of Risk Factors dimension, 0.88 for the Knowledge of Signs and Symptoms dimension, 0.81 for the Attitude to Breast Cancer Prevention dimension, 0.78 for the Barriers of Breast Screening dimension, and 0.61 for the Health Behavior related to Breast Cancer Awareness dimension. The factor loadings of the scale items, the eigenvalues of the obtained structures and the variance values explained are presented in Table 3, while the corrected total item correlation for the items under each factor and the Cronbach Alpha value of the scale if the item is deleted are presented in Table 4.

Table 3: Exploratory factor analysis results of breast cancer awareness scale

Table 4: Corrected item total correlation values for scale items and cronbach alpha values of the scale if the item is deleted

In this study, correlations between factors were also examined. The results revealed that there is a positive relationship between the dimensions of Knowledge of Signs and Symptoms, Knowledge of Risk Factors, and Barriers of Breast Screening. On the other hand, the Attitude to Breast Cancer Prevention dimension was found to have a negative relationship with the dimensions of Knowledge of Signs and Symptoms and Knowledge of Risk Factors. Furthermore, a negative relationship was revealed between the Barriers of Breast Screening and the Health Behavior related to Breast Cancer Awareness dimensions (Table 4, 5).

Table 5: Correlations between factors

Discussion

With this study, the validity and reliability analyses of the B-CAS were made and the scale was adapted to be used in the Turkish context. The B-CAS is the first tool known in Turkey to assess the awareness of breast cancer among women. The analyses revealed that the BCAS is a valid and reliable tool.

In our study, as a result of the analysis of the factors under which the items were loaded, it was determined that there was 1 item that was loaded on both factors and 5 items whose loadings were not sufficient. After these 6 items were removed from the scale, the analyses were repeated. As a result of the repeated analyses, a five-factor structure, which explained 57.85% of the variance and had factor loadings ranging from 0.63 to 0.82, was obtained. When the factors under which the items were loaded were analyzed, it was seen that the item "The history of breast cancer in the family" is under the dimension of Knowledge of Risk Factors in the original scale; however, in our study, the item was loaded on the factors of both Knowledge of Risk Factors and Knowledge of Signs and Symptoms. Similarly, it was observed that there were problems about the same item in the original scale[47] and in the study of Solikhah et al. (2017).[48] In our study, the items "How many days a week do you eat fresh vegetables?" "How many days a week do you do exercise or sports?" "Have you heard about the breast screening policy of the health-care personnel in your neighborhood?" "How often do you have clinical breast screening?" and "How often do you have a mammography?" in the Health Behavior related to Breast Cancer Awareness dimension were removed from the scale. In the study of Solikhah et al. (2017), the items "How often do you eat dessert" and "How often do you eat high-fat chicken, beef or duck meat" were removed. According to our findings, of the 35 items in the original scale, 29 items were kept in the Turkish scale. In the study of Solikhah et al.[48] (2017), 33 items of the original scale were included in the new scale. In our study, it was thought that it was more appropriate to remove the items with inappropriate loadings. All the items under the factors showed a distribution similar to the factors in the original scale.[47]

The total Cronbach Alpha internal consistency coefficient of the scale was calculated as 0.71 in our study. The total Cronbach Alpha coefficient of the original scale was found to be 0.86,[48] while it was calculated as 0.79 in the study of Solikhah et al.[48] (2017). The Cronbach Alpha internal consistency coefficients of the sub- dimensions of the scale are 0.87 for Knowledge of Risk Factors, 0.88 for Knowledge of Signs and Symptoms, 0.81 for Attitude to Breast Cancer Prevention, 0.78 for Barriers of Breast Screening, and 0.61 for Health Behavior related to Breast Cancer Awareness. The scale showed good internal consistency in accordance with the original version administered to Thai women (?=0.86, factor score=0.71-0.83).[48]

Our results revealed a positive relationship between the dimensions of Knowledge of Signs and Symptoms, Knowledge of Risk Factors, and Barriers of Breast Screening. The dimension of Attitude to Breast Cancer Prevention was found to be negatively correlated with the dimensions of Knowledge of Signs and Symptoms and Knowledge of Risk Factors. The Barriers of Breast Screening dimension was found to be negatively correlated with the Health Behavior related to Breast Cancer Awareness dimension. In the study of Solikhah et al.[48] (2017), while the Knowledge of Risk Factors subscale was positively correlated with the subscales of Knowledge of Signs and Symptoms, Attitude to Breast Cancer Prevention, and Health Behavior related to Breast Cancer Awareness; the subscale of Attitude to Breast Cancer Prevention was found to have a negative relationship with the Health Behavior related to Breast Cancer Awareness subscale.

Conclusion

In this study, the validity and reliability of the B-CAS were tested by adapting it to Turkish. The analyses revealed that the validity and reliability of the Turkish form of the scale was sufficient. The Turkish version of the B-CAS, including 29 items, can be used as a valid, reliable, and easy-to-use self-report measurement tool in the assessment of women"s breast cancer awareness. The scale can contribute to the development and evaluation of appropriate educational interventions to increase breast cancer awareness in Turkish women. Future studies should focus on verifying the B-CAS in addition to evaluating construct and criterion validity.

Peer-review: Externally peer-reviewed.

Conflict of Interest: The authors declare that they have no conflict of interest.

Ethics Committee Approval: The study was approved by the Necmettin Erbakan University, Meram Medical Faculty Hospital, Clinical and Non-Pharmaceutical Research Ethics Committee (No: 2018/1540, Date: 26/10/2018).

Financial Support: The authors declared that this study has received no financial support.

Authorship contributions: Concept - V.K.; Design - V.K., K.A.; Supervision - E.E.; Funding - V.K., K.A., E.E.; Materials - K.A., E.E.; Data collection and/or processing - V.K.; Data analysis and/or interpretation - V.K., K.A.; Literature search - V.K., K.A.; Writing - V.K., K.A., E.E.; Critical review - E.E.

References

1) WHO. Cancer; 2018. Available at: https://www.who. int/news-room/fact-sheets/detail/cancer. Accessed Jun 2, 2020.

2) T.C. Sağlık Bakanlığı. Türkiye kanser istatistikleri 2015. Available at: https://hsgm.saglik.gov.tr/depo/ birimler/kanser-db/istatistik/Trkiye_Kanser_statistikleri_ 2015.pdf. Accessed Jun 2, 2020.

3) World Cancer Research Fund; 2018. Breast cancer statistics. Available at: https://www.wcrf.org/dietand- cancer/cancer-trends/breast-cancer-statistics. Accessed Feb 10, 2020.

4) T.C. Sağlık Bakanlığı. Türkiye kanser istatistikleri 2012. Available at: https://hsgm.saglik.gov.tr/depo/birimler/ kanser-db/istatistik/ANA_rapor_2012sooonn.pdf. Accessed Jun 2, 2020.

5) DeSantis CE, Bray F, Ferlay J, Lortet-Tieulent J, Anderson BO, Jemal A. International variation in female breast cancer incidence and mortality rates. Cancer Epidemiol Biomarkers Prev 2015;24(10):1495-506.

6) Porter P. "Westernizing" women"s risks? breast cancer in lower-income countries. N Engl J Med 2008;358(3):213-6.

7) Özmen V, Özmen T, Doğru V. Breast cancer in Turkey; an analysis of 20.000 patients with breast cancer. Eur J Breast Health 2019;15(3):141-6.

8) WHO. Guide to administration and question by question specifications. Available at: http://www.who.int/ healthinfo/survey/instruments/en/index.html. Accessed Nov 30, 2011.

9) Breast cancer screening program national standards; 2018. Available at: http://www.kanser.gov.tr/dosya/ tarama/meme.pdf. Accessed Feb 10, 2020.

10) T.C. Sağlık Bakanlığı. Kanser taramaları. Available at: https://www.hsgm.saglik.gov.tr/tr/kanser-taramalari. Accessed Feb 10, 2020.

11) Akinyemiju TF. Socio-economic and health access determinants of breast and cervical cancer screening in low-ıncome countries: analysis of the world health survey. PLoS One 2012;7(11):e48834.

12) Tripathi N, Kadam YR, Dhobale RV, Gore AD. Barriers for early detection of cancer amongst Indian rural women. South Asian J Cancer 2014;3(2):122-7.

13) Unger-Saldaña K. Challenges to the early diagnosis and treatment of breast cancer in developing countries. World J Clin Oncol 2014;5(3):465-77.

14) Youlden DR, Cramb SM, Yip CH, Baade PD. Incidence and mortality of female breast cancer in the Asia-Pacific region. Cancer Biol Med 2014;11:101-15.

15) Mbuka-Ongona D, Tumbo JM. Knowledge about breast cancer and reasons for late presentation by cancer patients seen at Princess Marina Hospital, Gaborone, Botswana. Afr J Prim Health Care Fam Med 2013;5(1):465.

16) Ibrahim NA, Oludara MA. Socio-demographic factors and reasons associated with delay in breast cancer presentation: a study in Nigerian women. Breast 2012;21(3):416-8.

17) Espina C, McKenzie F, dos Santos-Silva I. Delayed presentation and diagnosis of breast cancer in African women: a systematic review. Ann Epidemiol 2017;27(10):659-71.

18) Azubuike SO, Celestina UO. Breast cancer: the perspective of northern Nigerian women. Int J Prev Med 2015;6(1):130.

19) Kendir C, Kartal M. Health literacy levels affect breast cancer knowledge and screening attitudes of women in Turkey: a descriptive study. Turk J Public Health 2019;17(2):183-94.

20) Rakkapao N, Promthet S, Moore MA, Solikhah S, Hurst CP. Assessing breast cancer awareness in Thai women: validation of the breast cancer awareness scale (B-CAS). Asian Pac J Cancer Prev 2017;18(4):995- 1005.

21) Linsell L, Forbes LJ, Burgess C, Kapari M, Thurnham A, Ramirez AJ. Validation of a measurement tool to assess awareness of breast cancer. Eur J Cancer 2010;46(8):1374-81.

22) Norlaili AA, Fatihah MA, Daliana NF, Maznah D. Breast cancer awareness of rural women in Malaysia: is it the same as in the cities? Asian Pac J Cancer Prev 2013;14(12):7161-4.

23) Ranasinghe HM, Ranasinghe N, Rodrigo C, Seneviratne R, Rajapakse S. Awareness of breast cancer among adolescent girls in Colombo, Sri Lanka: a school based study. BMC Public Health 2013;13:1209.

24) Liu LY, Wang F, Yu LX, Ma ZB, Zhang Q, Gao DZ, et al. Breast cancer awareness among women in eastern China: a cross-sectional study. BMC Public Health 2014;14:1004.

25) Sathian B, Nagaraja SB, Banerjee I, Sreedharan J, De A, Roy B, et al. Awareness of breast cancer warning signs and screening methods among female residents of Pokhara Valley, Nepal. Asian Pac J Cancer Prev 2014;15(11):4723-6.

26) Türkiye Halk Sağlığı Kurumu Kanser Daire Başkanlığı. Meme kanseri tarama programı ulusal standartları; 2013. Available at: http://www.kanser.gov.tr/Dosya/ tarama/meme.pdf. Accessed Nov 6, 2017.

27) Şencan H. Sosyal ve davranışsal ölçümlerde güvenirlik ve geçerlik. Ankara: Seçkin Yayıncılık; 2005. p. 190.

28) Çokluk ÖS, Şekercioğlu G, Büyüköztürk S. Sosyal bilimler için çok değişkenli istatistik: SPSS ve Lisrel uygulamaları. Ankara: Pegem Akademi Yayıncılık; 2012. p. 100-55.

29) Erdoğan S, Nahcivan N, Esin N. Hemşirelikte araştırma süreç, uygulama ve kritik. İstanbul: Nobel Tıp Kitabevi 2014. p. 216-31.

30) Büyüköztürk Ş. Sosyal bilimler için veri analizi el kitabı, Ankara: Pegem A Yayıncılık; 2007.

31) Sönmez V. Alacapınar FG. Örneklendirilmiş bilimsel araştırma yöntemleri. Ankara: Anı Yayıncılık; 2013.

32) Erefe İ. Hemşirelikte araştırma ilke süreç ve yöntemleri. İstanbul: Odak Ofset; 2012. p. 170-88.

33) Basham A, Sedlacek WE. Validity. In: American Counseling Association, editor. The aca encyclopedia of counseling. Alexandria, VA: American Counseling Association; 2009. p. 557.

34) Yıldırım A, Şimşek H. Sosyal bilimlerde nitel araştırma yöntemleri. 9th ed. Ankara: Seçkin Yayıncılık; 2013.

35) Ayre C, Scally AJ. Critical values for Lawshe"s content validity ratio. Measur Eval Counsel Dev 2014;47(1):79-86.

36) Kvale S, Brinkmann S. InterViews: learning the craft of qualitative research interviewing. Los Angeles, CA: Sage Publications; 2009.

37) Frank-Stromberg M, Olsen SJ. Instruments for clinical health-care research. London: Jones and Bartlett; 2004.

38) Wilson FR, Pan W, Schumsky DA. Recalculation of the critical values for Lawshe"s content validity ratio. Measur Eval Counsel Dev 2012;45(3):197-210.

39) Cohen RJ. A coefficient of agreement for nominal scales. Educ Psychol Measur 1960;20(1):37-46.

40) Davis LL. Instrument review: getting the most from your panel of experts. Appl Nurs Res 1992;5(4):194-7.

41) Lawshe CH. A quantitative approach to content validity. Pers Psychol 1975;28(4):563-75.

42) Lindell MK, Brandt CJ, Whitney DJ. A revised index of interrater agreement for multi-item ratings of a single target. Appl Psychol Measur 1999;23(2):127-35.

43) James LR, Demaree RG, Wolf G. An assessment of within-group interrater agreement. J Appl Psychol 1993;78(2):306-9.

44) Tinsley HE, Weiss DJ. Interrater reliability and agreement of subjective judgments. J Counsel Psychol 1975;22(4):358-76.

45) Veneziano L. Hooper J. A method for quantifying content validity of health related questionnaires. Am J Health Behav 1997;21:67-70.

46) Alpar CR. Uygulamalı istatistik ve geçerlik-güvenilirlik. 3rd ed. Ankara: Detay Yayıncılık; 2014.

47) Rakkapao N, Promthet S, Moore MA, Hurst CP. Development of a breast cancer awareness scale for Thai women: moving towards a validated measure. Asian Pac J Cancer Prev 2016;17(2):851-6.

48) Solikhah S, Promthet S, Rakkapao N, Hurst CP. Validation of an Indonesian version of the breast cancer awareness scale (BCAS-I). Asian Pac J Cancer Prev 2017;18(2):515-22.