Redefining assessment: Creating a groundbreaking prototype for domain affective in project-based learning

This research aims to design a prototype of affective domain self-assessment in project-based learning (PjBL). The design process of the instrument prototype in this study combines the aspects of the affective domain in Bloom ’ s taxonomy with PjBL stage. The design of this instrument is a self-assessment. The design method of the affective domain self-assessment instrument prototype for PjBL includes determining the objectives and specifications of the test, developing the framework, preparing the blueprint, and determining the measurement scale. The validity test used in the design of this instrument prototype is content validation conducted by experts’ validators from doctoral students in the field of educational research and evaluation and education management. Qualitative content validation using face validity to examine the appropriateness of the instrument prototype design with the purpose of the instrument developed. Secondly, quantitative content validity using Lawshe ’s content validity ratio (CVR) and content validity index (CVI). The results obtained showed that CVR value of the panelists was scattered from zero, where half of the panelists stated that the item was relevant, to one, where all panelists stated that the item was relevant, the distribution of CVR ratio was mostly at 0.75. CVI index price is 0.6083, which is very favorable.


INTRODUCTION
In the present era, where the development of science and technology is growing rapidly and research advances are increasingly sophisticated, academic institutions have tried to equip students with a combination of cognitive knowledge, and professional skills or what is commonly referred to as hard skills as well as non-technical skills such as problem-solving and teamwork or what is commonly referred to as soft skills (Guo et al., 2020).In the context of learning, strengthening soft skills and hard skills is done by facilitating the process of students constructing their knowledge so that it can be interpreted by students properly and is useful in its application in everyday life.This facilitation process will involve a phase of "transfer", which is defined as the utilization of acquired knowledge and skills in a new context, with different content or in a different situation from the original acquisition (Schunk, 2012).A particularly student-centered transfer phase can stimulate learners' creativity and enable learning to be more meaningful.One such way is through project-based learning (PjBL).PjBL can be briefly defined as educational education that involves the acquisition of knowledge and skills through the process of producing and completing a project (Sánchez-García & Pavón-Vázquez, 2021).Various studies have been conducted using PjBL as a learning method for skill development (Rohm, 2021), creative problem solving (Chen & Chan, 2021), student attitudes (Parker, 2020), interpersonal competence development (Crespí et al., 2022), project-based earning with interdisciplinary approach (Hart, 2019), collaboration in PjBL (Hussein, 2021), group work orientation through PjBL (Jaiswal et al., 2021), students perception in psycho affective through PjBL (Sánchez-García & Pavón-Vázquez, 2021), and so on.Furthermore, Sánchez-García and Pavón-Vázquez (2021) explain that PjBL presents a new approach in pedagogy that emphasizes collaboration and cooperation among students.This approach fulfils all the criteria necessary to be considered a valuable instructional technique.The common assessment involves the assessment of students' learning outcomes; however, PjBL also promotes students' personal and professional growth by enhancing their transversal competencies through skills acquisition and development (Crespí et al., 2022).What if the assessment is a process carried out by the learners themselves and conducted at each stage of PjBL?What if they reflect on the learning, they have gone through in each stage of PjBL?This question is the identification point of the problem addressed in this research.
Furthermore, the characteristics of PjBL include inquiry-centered emphasis on educational objectives, engagement in pedagogical activities, cooperation among students, integration of supportive technologies, and production of tangible outcomes (Guo et al., 2020).These characteristics create a process that involves various learning resources and participant collaboration.This process is the inspiration to develop an assessment prototype in the affective domain of PjBL, especially at the university level.The instrument used is the self-

OPEN ACCESS
assessment instrument with Likert scale (one-four) (Johnson & Morgan, 2016).Self-assessment was selected because student-centered assessment is indispensable in reinforcing the importance of the assessment process, especially regarding one's own competence (Setiawan et al., 2019).
Based on the explanation above, the research questions posed are, as follows: 1. How to design effective self-assessment instruments for each process in PjBL?
2. How to validate the content of the designed instrument prototype so that it is suitable for its purpose?
The purpose of this research is to design a prototype of a selfassessment instrument in PjBL process in the affective domain to assess the attitude of students in the learning process.This research is an initial prototype design stage so that at the prototype stage the steps will be discussed up to the results of content validation both quantitatively and qualitatively carried out by experts to ensure the suitability of the instrument content with its objectives.Novelty promoted through this research, this research promotes the innovative ways to measure changes in students' attitudes in each stage of PjBL adapted from the stages of PjBL by (Crespí et al., 2022).So that the results could contribute to provide information for educators to evaluate their students' achievements, support the implementation of meaningful learning, and help educators to evaluate PjBL process that observes changes in students' attitudes through their self-assessment process in each process.In addition, this study focused on content validation.Content validity refers to how relevant and representative the elements of the assessment instrument are to the specified construct or purpose of the assessment (Yusoff, 2019).

Research Design
The self-assessment instrument design process in this study follows the methodology of Istiyono (2020), Kalkbrenner (2021), and Shrotryia and Dhanda (2019), where the instrument design process is carried out, as shown in Figure 1.
In this research, the emphasis is on designing the prototype (beta version) so that the design process is limited to expert validation to ensure the suitability of the content designed with its function.The design of the affective domain PjBL self-assessment instrument prototype is carried out in accordance with the process of implementing PjBL itself by adapting the stages of implementing PjBL by Crespí et al. (2022) with the distinction in the initial process, there is a project briefing by the educator to stimulate students' knowledge construction.The stages of PjBL consist of the following stages: 1.The first stage of the project is to analyze, conduct research, and develop proposals both individually and in teams.
At this stage, students are required to engage in the process of observing the phenomena around them and identifying specific needs that they want to address collaboratively as a team.
2. The second stage involves project design and development.
During this stage, teams utilize various tools to formulate and define their project proposal.
3. The third stage is application.
Teams implement their projects over a period of about 16 weeks equivalent to one semester.Here students enhance their competence in areas such as team management and conflict management.
4. The fourth stage is the submission of the final report.
The report contains a reflective compilation of the project raised and analyzed.

The fifth phase is project presentation.
In this phase the team showcases their project with a focus on the analyses conducted.
Based on the above stages of PjBL, it will then be related to the levels of the affective domain of Bloom's taxonomy (Hoque, 2016;Metfessel, 1969) to bring up relevant keywords so that it can serve for students to assess their competence at each stage of PjBL including the steps in Table 1.
In this affective domain learning outcome assessment, a scale is then determined to measure learners' self-assessment of their attitudes to PjBL process.learners' self-assessment related to their attitude in PjBL process.The determination of the scale on the assessment serves to provide instructions for the measurement of a statement in a survey (Johnson &   Morgan, 2016).In the following self-assessment, the scale used is a Likert scale with four filling options.

Data Analysis
The prototype of self-assessment instrument for PjBL was analyzed using content validation.The first content validation analysis was qualitative using face validation (Dolatkhah et al., 2020).In the face validation, expert raters reviewed the grammar, clarity of wording, feasibility, and suitability of the instrument for its purpose (Dolatkhah et al., 2020).Finally, face validity is conducted to evaluate the feasibility, understandability, and clarity of vocabulary, format, and presentation and then results will be revised according to their suitability and necessity (Dolatkhah et al., 2020).Then, analyzing quantitatively using content validity ratio (CVR) method by Lawshe (1975) this approach by Lawshe (1975) to assess substantive validity of a measure, shows that higher values correspond to greater substantive validity for each item.
Content validity index (CVI) thereafter indicates extent of perceived congruence between an individual's ability to perform in a particular task area and their performance on the test under examination (Anuar & Sadek, 2018;Lawshe, 1975;Yusoff, 2019).In this research, the term "essential" was replaced with the term "relevant" to emphasize the item's fit for purpose.
CVR and CVI calculations were conducted using the formula (Anuar & Sadek, 2018;Lawshe, 1975;Yusoff, 2019), as follows:  = , where Ne is the number of panelists stating relevance and N is the number of panelists.

RESULTS
The design results of the self-assessment instrument for the affective domain in PjBL are, as follows:

Developed an instrument matrix/framework
In the design process of PjBL affective instrument prototype, a combination was carried out by adapting the stages of PjBL (Crespí et al., 2022) with the affective domain from Bloom's taxonomy (Hoque, 2016;Metfessel, 1969) (Table 3).

Developing instrument blueprint
Table 4 shows development of instrument prototype blueprint.

Content & scale development
Table 5 shows sample of designed instrument prototype.

Face validity
Table 6 shows face validity by experts.

Content validity
Table 7 shows content validation from experts.At the final product/prototype presentation stage, the experience I can take is prioritize the time management part of the presentation.

1-4
At the final product/prototype presentation stage, the experiences I can take away are focus on the developed project product.

1-4
At the final product/prototype presentation stage, the experience I can take away is only needed to make sure the presentation slides are beautiful, and the content is as simple as possible.

1-4
At the final product/prototype presentation stage, the experience I can take is the presentation flow packaged the entire process of working on the project task attractively.

1-4 30 26
The significant changes I feel after completing this project are opportunities for problems to be researched.

1-4
The significant changes that I feel after completing this project are more confident to present my opinion in group discussions.

1-4
The significant changes that I feel after completing this project are more prepared and challenged in the implementation of the next project.

1-4
The significant changes I felt after completing this project were enjoyed process of developing this project due to its usefulness & meaningfulness.

DISCUSSION
Measurement is the process of systematically quantifying individuals to represent individual traits (Lester et al., 2014).This research was conducted to design a prototype of affective domain measurement instrument at each stage of PjBL.Measurement instruments are useful as tools to collect, evaluate, and use evidence of learner learning for various purposes (Brookhart & McMillan, 2019).The design of the affective domain assessment instrument prototype in PjBL is carried out by combining two things, firstly PjBL stages adapted from (Crespí et al., 2022) including the introduction stage, project design, application, report preparation, and presentation of report results.All these stages are unique and have processes that can stimulate the meaningfulness of the learning process of students.This process is then combined with the affective domain of Bloom's taxonomy (Hoque, 2016;Metfessel, 1969), which is through the most basic level of "awareness" to "internalization".
The first stage of prototype design is to determine the specifications of the prototype instrument being designed.Prior to instrument development, objectives must be clearly defined by articulating the specific constructs to be measured (Kalkbrenner, 2021).In this research, the prototype instrument designed is a self-assessment instrument to measure the attitude of students in participating in PjBL process.The aspects measured are the affective domain at each stage of PjBL implementation to assess changes in students' attitudes.Based on these specifications, an instrument matrix/framework is developed, which is a process, where the researcher is responsible for selecting a theory or several theories and or consolidating existing literature findings to establish an empirical framework for the design process of this instrument prototype (Kalkbrenner, 2021).Based on the matrix results in Table 3, the preparation of the matrix in this study by combining aspects of Bloom's taxonomy affective domain, namely receiving, responding, valuing, organization, and characterization by value or complex value.This aspect is then reduced to sub-aspects such as the example of the Receiving aspect is reduced to the sub-aspect of willingness to respond and control selected attention (Hoque, 2016;Metfessel, 1969) the determination of these sub-aspects is adjusted to the stages of PjBL by (Crespí et al., 2022).For example, in the first stage of PjBL "introduction" the sub-aspect considered appropriate based on the definition of this stage is "receiving" with the sub-aspect "willingness to respond" and "control selected attention" because at this stage it is still a process to recognize and identify problems based on the explanation of the project task delivered by the teacher, and so on.After that, it was also determined that the distribution of item numbers was evenly distributed and not in order only in certain aspects.
In the design of this affective instrument prototype, the selfassessment form was chosen because it learnt the meaningfulness of PjBL, where PjBL activities require students to engage in exploration, negotiation, interpretation, and creation to develop solutions, as well as implementation.This facilitates the transfer of data, information, and knowledge in a less formal and more transparent way.Most interestingly, students are encouraged to develop critical thinking skills, including the stages of analysis, and synthesis (Chen & Chan, 2021;Crespí et al., 2022;Parker, 2020).Thus, the whole process is intended to elicit a transformation in students' attitudes and behavior towards knowledge (Parker, 2020).This is the main inspiration for the design of the affective instrument prototype in PjBL.Self-assessment in the design of this instrument prototype is designed for the reflective process of students at each stage of PjBL process.Thus, at the end of the process the teacher will obtain complete information related to the transformation of students' attitudes during the learning process.
In the matrix design of this instrument prototype shown in Table 3, the end of the process is the presentation and reflection of students after going through a series of learning stages with PjBL.So, the level of Bloom's taxonomy of the affective domain that is achieved is the first organization with the sub-aspect of organization of a value system (Hoque, 2016;Metfessel, 1969) the focus of item development is that students assess their ability to organize their preparation for presenting project results.Then at the next level, namely reflection, the aspect of the affective domain that is targeted is characterization, namely learners reflect on the entire series of PjBL that has been passed to draw lessons learned.
The next stage is blueprint development.Kalkbrenner (2021) states that the use of theoretical blueprints can serve as a valuable means of increasing the content validity of a measure, which provides benefits for researchers by allowing the creation of content and domain areas related to measurement constructs.Also, it facilitates determining the approximate proportion of items that should be created for each content and domain area.Table 4 shows the blueprint design for this affective PjBL instrument prototype.Through Table 4, it is observed that at this stage, the placement of item numbers and the approximate scale used as well as the description of the items that will be used to measure learners' attitudes have been determined.In the design of this instrument prototype, the measurement scale chosen is the Likert scale, so that the lowest score is one in the never option column, and the highest score is four in the always option, but for negate statements, the score will be the opposite (Johnson & Morgan, 2016).In this Blueprint, the statement sentences are arranged using also the keywords contained in Bloom's taxonomy (Hoque, 2016).
After the Blueprint is compiled, the next step is to arrange the instrument in a complete format along with the rating scale to be filled in.If this process has been carried out, then the next is the process of ensuring the suitability of the content design with the purpose of the measurement from the expert or called content validation or expert validation, where the validating pane or validator is an individual who has expertise in the relevant field.The expert panelists involved in the design of the affective domain self-assessment prototype for PjBL are 8 doctoral students consisting of doctoral students of educational research and evaluation and educational management, the experts involved are seven females and one male who are all from Yogyakarta State University, Indonesia.A selection of doctoral students from the educational research and evaluation and educational management programs were qualified to be assigned as experts in content validation due to their extensive training and expertise in these areas.They have been trained in research methods, program evaluation, and policy analysis, which enables them to critically analyze and evaluate the content of this instrument.They have also gained practical experience through research assistantships, internships, and working with faculty on research projects.This hands-on experience, combined with their theoretical knowledge, makes them well suited to provide expert judgment on the content of this prototype instrument.
The qualitative analysis process is done with face validity.The instrument review process through face validity aims to review the suitability of question items with indicators; grammar; statement bias; suitability of instrument format; and logical sequence of instruments (Istiyono, 2020).The results in Table 6 shown that there were several parts that needed to be revised or improved, namely in terms of Indonesian grammar, the use of more effective sentences, the use of clearer sentences to be able to reflect the aspects to be measured, as well as several writing errors that needed to be corrected to avoid reader misunderstanding.The aspect of language used is very important to produce materials that can be understood by individuals participating in the test (Johnson & Morgan, 2016).Therefore, readjustments were made to ensure that the sentence structure was more effective and easier to understand with the purpose of measurement.
The next step is quantitative analysis using Lawshe's (1975) CVR technique.This method is carried out to assess substantive validity, a measure indicating that higher values correspond to greater substantive validity for each item (Anuar & Sadek, 2018;Lawshe, 1975).The results of the content validity test with CVR conducted by eight expert validators show that CVR value is spread from zero to one.As shown in Table 7, CVR results indicate there is one item that has a CVR value of zero.This is in accordance with the explanation (Lawshe, 1975) that when half the panelists say, "yes or relevant" and half "no" then CVR value will result in zero.Items with CVR scores above 0.5 will immediately pass for use, while items with scores below 0.5 will be reviewed and matched with panelist comments on each item then it will be decided whether the item is revised or eliminated.CVR results were then averaged to obtain CVI value.After determining which items should be included in the final form, CVI for the entire test was calculated.According to (Lawshe, 1975), CVI is merely the average of CVR scores for the retained items.Operationally, CVI is the average proportion of overlap between the test items and the domain of job performance.
Table 7 presents the results of CVI calculation, which is 0.6083.Based on Lawshe's (1975) explanation, it is known that the closer to one, the better the validity of the instrument content.So, when analyzing the results obtained on this prototype instrument, it can be assumed that its content validity is moderate.CVI of 0.6083 indicates that the content of the instrument is quite relevant to the intended construct, but there is still room for improvement.To improve the content validity of the instrument, it is possible to revise and retest items that do not meet the content validity threshold or delete items that do not show acceptable content validity.It should be noted that the use of CVR and CVI as quantitative metrics to assess the validity of simulation evaluation tools, which are intended for users and researchers (Anuar & Sadek, 2018).
In general, the results of the face validity assessment show that the self-assessment items need to be revised in terms of grammatical and sentence efficiency.some items such as number 1 and number 2; 19 and 21; 22, 23, and 25 is almost the same, one can be chosen.other inputs based on expert validator reviews such as look for a more efficient sentence, pay attention to typographical errors, statement presentation format needs to be revised to make it easier to understand, consider using more concrete verbs.The results of validation using CVR showed 25 instrument items got the number of agreement rates above five, which ranged from six-eight, where CVR results were 0.5-1.these items passed to be used.Items 22, 23, and 25 have sentences that are almost the same so that in line with the results of face validity these items are recommended to be selected.As a result, item number 22 was dropped and for item 23 and item 25 were used for the actual instrument.Similarly, for item number 19 and item number 21, item number 19 was dropped.
The items in Table 4 are statement items that have undergone revision.It can be seen in Table 4, that in the item number column there were originally 30 self-assessment items.After going through content validation of both item selection through the results of face validity recommendations and prices from CVR experts, there are finally 26 final items for this PjBL self-assessment prototype.Finally, design process of affective instrument prototype for PjBL is designed to not only benefit teachers in conducting the measurement process, but it also promotes relevant learning theories such as meaningful learning, where effective learning requires active involvement of learners, coupled with introspection and self-assessment and belief in importance of the learning process (Schunk, 2011).

CONCLUSIONS
Based on the results of data analysis and the objectives of this study, the design of the prototype of the affective domain assessment instrument should pay attention to aspects of a good instrument design process such as setting test objectives, compiling a framework or framework, compiling blueprints, and scoring scales.In addition, the preparation of effective and efficient statement sentences is also very essential in the design process of this prototype instrument.The results of expert validation support that good sentence structure will increase the relevance of the purpose of measurement through this instrument.
This instrument is expected to be refined from its prototype version in the future.This instrument offers an innovation as a tool to monitor the track record of students' attitude transformation in constructing their learning experience in this PjBL.This instrument is also expected to be utilized by educators as a tool for monitoring and evaluating students' learning stages.
This instrument is a beta version that still has room to be improved both in terms of language quality and flow design.

Figure 1 .
Figure 1.Design prototype instrument process (Adapted from Shrotryia & Dhanda, 2019) I have the awareness to fulfill my responsibilities according to the division of tasks in the team.presentation of my results by prioritizing the prototype development process part of the project.1-4I prepared the presentation of the results by prioritizing the data analysis part of the project prototype trial.
of items; R: Revision; S: Scale; & TI: Total items

Table 3 .
Matrix development

Table 4 .
Development of instrument prototype blueprint

Table 5 .
Sample of designed instrument prototype

Table 6 .
Face validity by experts