Broadening the benefits of PBL: a “good” problem

Copyright © 2018 REPEC. All rights, even translation, are reserved. It is allowed to quote part of articles without prior permission if the source is identified. cc BY Published in Portuguese and English. Original Version in Portuguese. Abstract: This study addresses the characteristics of a “good” problem for a management accounting course that applies the Problem Based Learning (PBL) method, in which undergraduate students are expected to define their own problems. In this case, PBL is applied as an integrative discipline, from the middle to the end of the course. The innovation is to take advantage of students’ practical knowledge and context and to expand the potential of PBL, including the identification and design of a problem, as well as the solution supported by the literature. The empirical part takes place in class groups of students, using the “action research” methodology. We particularly focused on the intrinsic and utility characteristics of good problems of 17 groups that attended the course in 2014 and 2015. The main implications of the study are: (i) highlight an opportunity to broaden the potential benefits of PBL by better characterizing a good problem for the PBL approach; (ii) discuss critical issues for PBL which are different from the traditional approach; (iii) use the eleven characteristics for a good problem in PBL in a segmented manner; and (vi) provide evidence that the role of the professor requires adaptation due to the level of uncertainty this approach encourages.


Introduction
Previous scholars from the educational field have been suggesting that Problem Based Learning (PBL) can address the organizational demands in terms of professional knowledge, skills and attitudes to reduce the distance between theory and practice (Stanley & Marsden, 2012;Milne & McConnell, 2001;Breton, 1999). The PBL approach consists in three pillars, which are the problem, the student, and the teacher (Majoor et al., 1990;Schmidt, 1983;Savery, 2006). In this paper, we focus on the problem pillar, as it can be considered a vehicle to integrate theoretical discussions into the business reality (Duch, Groh, & Allen, 2001;Hmelo-Silver, 2004;Hansen, 2006;Savery, 2006). Problems should be based on real-life situations identified in practice, so that they represent authentic contexts and are related to the future roles the students are to perform within the job market (Hallinger & Lu, 2012).
The context of this study is connected to the area of business and focuses on an undergraduate accounting course with two distinct branches, which are financial accounting and management accounting (Horngren, Foster & Datar, 2000;Frezatti, Aguiar, & Guerreiro, 2007). The study is developed in an integrative course in the area of management accounting, favoring topics such as strategy, budgeting, costs, performance indicators, information systems and controlling, that is, the typical elements in an organization's management process (Anthony et al., 2014).
Valuing what students know (Schmidt, 1983;Branda, 2009;Hmelo-Silver, 2004;Zwaal & Otting; and reinforcing the need for conceptual knowledge were the targets in developing and designing the course that is the subject of this study (Martins & Frezatti, 2015;Halliger & Lu, 2012;Hung, 2006). Here, the students, by means of self and independent learning, study concepts again depending on need and interest, in accordance with the problems developed in their own groups (Schmidt, 1983;Hmelo-Silver, 2004;Dochy, Segers, Bossche, & Gijbels, 2003;Zwaal & Otting;Scott, 2014). Thus, more than proposing the solution to problems, the course proposes to develop the student's ability to identify a relevant problem, structure it in an intelligible format and subsequently propose a reasonable solution.
The gap to be addressed involves how to broaden students' abilities through the choice of subject matter, the structuring of cases and problems, and the solutions to the problems, in a non-homogeneous environment in terms of interest and conceptual knowledge, given that the students' professional experience can derive from working as an intern, as a junior employee, a manager, a director or even a business owner. If on the one hand the environment is rich, on the other, the challenge is in providing guidance, orientation and boundaries to the development of problems that can involve a variety of themes and approaches.
A situation in which students have professional experience, involving knowledge that has been or will be presented in the educational environment, providing freedom for them to be able to identify a relevant problem to be addressed in the classroom within their context of professional experience creates an opportunity to bring real life to the classroom. At the same time, however, this reduces the "controllability" of the environment for the professor. The group's choice of problem brings a very strong degree of responsibility and constructivism to the classroom.
Despite being very rich, this situation creates an enormous problem for teachers to understand and evaluate problems in a way that is perceived as relevant and similar in terms of difficulty and contribution. They need to have tools at their disposal that permit discussing and guiding the elaboration and structuring of a problem that is relevant and part of the students' experience, which is essential so as not to undermine the whole model for developing PBL. Not accepting a problem, using arguments that are not very "objective" and perceived as being "to the teacher's taste and preference", may cancel the potential benefits of the students' contribution, primarily when the students' interests are not aligned with the course objectives. On the other hand, accepting a problem that does not result in the benefits of the approach simply to motivate the student is not seen as appropriate. Fábio Frezatti, Daniel Magalhâes Mucci, Daiana Bragueto Martins The design of the problem in the management accounting area is developed based on the possibility of the student experiencing it in an organization. After identifying the problem, this is legitimized when all the group members consider it to be of interest. Thus, the problem chosen comes to be of interest from the group's perspective and does not necessarily meet the learning objectives of the discipline and of the course's pedagogical project, nor of the institution. For this reason, a clear approach that allows the teacher to analyze, judge and assume a position regarding the appropriateness and quality of the problems, from the "inside out", is essential.
The problem is the most "objective" element when there is a clear definition of need and utility for the group that will apply it. Having a good problem involves context (Hung, 2006;Hallinger & Lu, 2012), its relevance to society (Hmelo-Silver, 2004;Zwaal and Otting, 2015;Ribeiro, 2008;Hung, 2006;Hansen, 2006;Savery 2006), and its authenticity in relation to the issues (Hallinger & Lu, 2012;Scott, 2014). In medicine, the word context has more to do with the way the problem is directed (Dolmans, Snellen-Balendong, & Van der Vleuten, 1997). In accounting, then, the word is strongly associated with the way in which the student lives and develops (Stanley & Marsden, 2012;Milne & McConnell, 2001;Breton, 1999).
Therefore, based on the framework presented by Sockalingam and Schmidt (2011) about the characteristics of a "good" PBL problem, this paper discusses the following research question: how and to what extent can analyzing and validating student-designed problems in a management accounting PBL course be effective?
The purpose of this study is to analyze and discuss the intrinsic and utility design characteristics of good PBL problems that are proposed by groups of students in a management accounting course that applies the PBL approach. Therefore, we evaluate the problems proposed by the students to investigate whether these problems attend to the characteristics the literature suggests as relevant to characterize an effective problem for the PBL approach.
As contributions, it is understood that this study comes to add to the lack of research discussing the design of a problem (features and functions), within the learning objectives proposed by PBL (Barrows, 1986;Duch, 2001;Hung, 2006;Jonassen & Hung, 2008;Dolmans et al., 1997;Sockalingam & Schmidt, 2011). Moreover, it expands the usefulness of the method in a skill valued in the field of management accounting, in which discussion of the PBL methodology has become more intense (Stanley & Marsden, 2012;Milne & McConnell, 2001).
A contribution from the practical point of view, via the use of the Sockalingam & Schmidt's (2011) approach, is the increase in the potential of professors' assertiveness regarding the use of problems brought by students, using the guidance proposed in the study. This is unlike other areas, in which the problem is developed by professors, such as medicine (Dolmans et al, 1997) and microbiology (Sockalingam & Schmidt, 2011). Lastly, it legitimizes collaborative work in integrating course objectives with individual and student group expectations.

Literature Review and Theoretical Framework
The PBL teaching-learning process consists of three fundamental elements: the problem, the student, and the tutor (Majoor et al., 1990). The problems are essential and are seen as the "heart" of the PBL method (Hung, 2006), as they trigger the whole teaching-learning process (Sockalingam and Schmidt, 2011). The literature has shown that a problem's design can be inappropriate to achieve the PBL objectives (Hung, 2006).
Recently, different authors have proposed models to address this gap in the literature, among which we highlight the seven tips by Dolmans et al. (1997), the 3C3R (Hung, 2006), and the eleven characteristics proposed by Sockalingam and Schmidt (2011). Dolmans et al. (1997, pp. 185-186) present 7 (seven) principles for the effective development of a problem in PBL, which we list next: (i) the content of the problem should adapt well to the students' prior knowledge; (ii) it should contain various suggestions that stimulate the students to elaborate on and develop; (iii) it should cover a relevant context for the future profession; (iv) it should propose basic concepts to encourage the integration of knowledge; (v) it should encourage self-learning; (vi) it should increase the students' interest in the course, underpinning discussion about possible solutions and/or alternatives; (vii) it should be aligned with the course objectives.
Broadening the benefits of PBL: a "good" problem The model discussed by Hung (2006) divides the components of a good PBL problem into main components (features) and cognitive process components, or actions the problem generates. The main components are content, context and connection. Content covers the knowledge developed via the scope and depth of the problems. The problem should include a valid context for the area of knowledge and be of depth. The possibility of connection would be the link between the content and context. The expected cognitive processes are research, creating a line of thought, and discussion. In this sense, the problem should favor self-directed student research, allowing students to develop a line of reasoning to discuss the problem. Finally, it should direct a process of synthesis, which integrates the lessons the students learned. Sockalingam and Schmidt (2011) characterized a good PBL problem from the perspective of students of microbiology. The students were asked: "What is your perception of a good problem and why?" Based on the analysis of the results, the authors presented eleven PBL problem characteristics, which they divided into 2 (two) groups called features and functions.
The features are elements of problem design, such as format, clarity, familiarity, difficulty and relevance, while the functions consist of expected potential results from working with the PBL problem, such as the link between the problem and the learning objectives, the problem arousing interest, stimulating critical analysis, promoting self-learning, stimulating elaboration, and promoting group work (Sockalingam & Schmidt, 2011). The model proposed by Sockalingam and Schmidt (2011) was used as a base for the purposes of this article.
Considering Sockalingam & Schmidt (2011)'s research, some elements are considered to compare, analyze for the sake of validation and consolidate what students consider relevant for the suitability of a problem from the PBL perspective.
The starting point are the 11 elements (Sockalingam & Schmidt, 2011), many of which are common in various contributions found in the literature (Scott, 2014;Zwaal and Otting, 2015;Mühlfelder, Konermann & Borchard, 2015;Martins & Frezatti, 2015;Martins & Espejo, 2015). The two different types of characteristics are separated and are: (i) those intrinsic to the problem and (ii) utility characteristics.
The intrinsic features of the problem (Table 1) should be found within the problem and can be affected by the problem developers, who analyze, understand and perceive them. For some of them, it is relatively easy to identify and express the actions needed to improve them. For others, this is more difficult given the degree of subjectivity and complexity to operate the analysis, understanding and improvements. They are: (i) familiarity with the problem, (ii) difficulty of the problem, (iii) relevance of the problem, (iv) clarity of the problem, and (v) format.
The investigated authors (Sockalingam & Schmidt, 2011) did not indicate a quantitative approach to address the issue and a rubric was created to carry out this evaluation. The literature does not specify operational criteria to determine each of the features for validating problems designed by the students. Thus, the approach of Brodie and Gibbings (2009) was considered in structuring the rubric laid out in Table 1. As shown in the analysis, however, placing the problem identified by the students into a model with quantitative specifications is very complex and in some cases even unfeasible, primarily concerning the utility characteristics.
The relationship between the features takes into account that format and familiarity provide the clarity for the students to understand the problem. Format can be addressed in the tutor sessions, but familiarity depends on the background experience with the discipline, i.e. being conceptual and acquired in the classroom during previous disciplines and/or experience in organizations, which is not feasible during the PBL course itself. As a result of this combination, the students will perceive the difficulty of the problem and the relevance for the participants' context in the plenary group. Difficulty can motivate as well as discourage, depending on the effort the participant finds necessary. In an environment of great difficulty, the perceived relevance of the problem may not be properly understood. A very low level of difficulty can attract students' attention as an activity to be overcome, but not as a challenge that provides relevant additions to knowledge. These elements together have an impact on the utility characteristics (Sockalingam & Schmidt, 2011). Fábio Frezatti, Daniel Magalhâes Mucci, Daiana Bragueto Martins In turn, the utility characteristics of the problem will influence the students. They are influenced by the intrinsic characteristics, can be listed and tend to indicate how intensely (Sockalingam & Schmidt, 2011) the problem: (i) links in with the learning objectives; (ii) arouses interest; (iii) stimulates critical analysis; (iv) promotes self-learning (v) stimulates elaboration; and (vi) promotes group work. Similar to the intrinsic characteristics, the operationalization of the utility characteristics of the problem is presented in Table 2.
In the way it was presented by Sockalingam & Schmidt (2011) there exists a "causal" relationship between the intrinsic and utility characteristics of the problem. This means that, in order to achieve results and enjoy benefits, the quality of the problems used in PBL is linked to the fact that they can have particular properties and characteristics (Van Berkel & Schmidt, 2000;Zwaal & Otting, 2015). It should be emphasized that, in the work by Sockalingam & Schmidt (2011), the weighting of importance was indicated from the students' viewpoint.

Concept:
Forms part of the students' experiential knowledge and is inserted into the seven topics present in the specification of possible sub-issues.

How to address:
Mapping of issues required to analyze the problem. Application of the diagnostic questionnaire and proof of knowledge mapping.
They had been exposed to all of the concepts required in the discipline They had been exposed to more than one concept required in the discipline They had not been exposed to any of the concepts required in the discipline Clarity of the problem Sockalingam & Schmidt (2011, 12); Schmidt and Moust (2000); Van Berkel & Schmidt (2000).

Concept:
Adaptation of what we are referring to is comprehension by others besides those who structured the problem.   (1997).

Concept:
The problem is related to day to day occurrences, that is, it is applicable or of use.

How to address:
Analysis of degree of reality contained in the problem, with it being found in many "relevant" organizations.
The problem reflects a real day-to-day organizational situation regarding at least one management accounting issue.
The problem partially reflects a real day-to-day organizational situation regarding at least one management accounting issue.
The problem does not reflect a real day-to-day organizational situation regarding at least one management accounting issue.

Concept:
There is the potential to seek various alternatives for solutions.

How to address:
The search for alternative solutions is considered feasible.  .

Concept:
Given balanced difficulty, the students can individually carry out self-learning.

How to address:
Research or interviewing specialists is needed to search for the answer.
They presented references different from those supplied by the teacher; they carried out interviews with specialists and visits to an organization.
They presented few references and/or did not carry out interviews with specialists and/ or visits to an organization.
They only used the material made available by the teacher.

Concept:
As long as it is easy to understand, it will arouse interest and can be quickly solved How to address: The solution itself indicates this. Giving up indicates the opposite.
They presented a creative and innovative solution to the problem that can or cannot be applied in practice.
They presented a partial solution to the problem, but it cannot be applied in practice.
They did not present a solution to the problem.
Broadening the benefits of PBL: a "good" problem

Intrinsic characteristics
Reference Criteria

Concept:
Team working to resolve the problem How to address: Visibility of teamwork

Research field
The institution was chosen because (i) it is interested in active learning processes, (ii) it provides professors with the freedom to provide new approaches to the learning process, (iii) it joins students with a high-level educational background, permitting the adaptation of styles during the course. Thus, we are concerned with a case in which both the individual students and the particular groups are focused on.
The course in this study is optional and offered in the evenings with the mission to integrate knowledge offered in several prior disciplines. It comprises 32 classroom hours, divided into 16 meetings with the students held exclusively in class (all persons in the same room). The structure of the classes involves division into six blocks: (i) hybrid classes (expository and group work), (ii) tutorial sessions, (iii) sharing of partial results, (iv) sharing of final results via films presented on Youtube, (v) a knowledge test and (vi) self-evaluation and peer-evaluation.
Even though the course has been offered on five occasions, the problems developed in the last two editions (2014 and 2015) were chosen for analysis, as the course design and procedures were applied in a more homogeneous way.
Regarding the learning objectives communicated to the students, these were: to apply learning via the identification, analysis and proposal of solutions for problems arising in Brazilian companies involving the field of management accounting, by means of group projects.
It is worth noting that, while the professor defined the themes and content, the students were responsible for selecting the real-life problems and the teams for elaborating them. When the students identified the problems in the companies where they worked, these were categorized according to Ribeiro (2008) as problems that demand real solutions for real people or organizations.

Problem
In this course, the problem is a central point that interlinks the academic world of theories and scientific research with professional practice, represented by the job market. Each group identified, analyzed, discussed and solved an authentic management accounting problem that was drawn in its original form from the professional reality of a Brazilian company (Araujo & Arantes, 2009). As each student identified a problem, and considering that each group contained at least three and at most six members, each team chose only one of the problems, by means of legitimization, to be constructed and elaborated throughout the course. From this perspective, it can be observed that the student departs from the context and then creates a problem.
It should be highlighted that neither the professor nor the students knew which team worked on what problem, which represented a challenge for both to direct the whole teaching-learning process of the course. Thus, it is observed that the course focuses on the context of each group and values not only knowledge, but also attitudes and skills.
The solution to the problem was discussed by means of tutorial sessions, which stimulated the exchange of background knowledge, knowledge acquired via independent self-learning, and knowledge obtained from interviews carried out with key people in the organizations. Thus, the construction of knowledge takes place via reflection, dialogue and the exchange of experience between the teacher and the student, in which both share their life experiences. The solution proposed for the problem was presented via scientific reports, including theoretical reflections and analyses of the problem studied. Each team presented both the partial (this took place in the middle of the course) and final results (in the last classes) to the whole group. These final results were exhibited through the creation of a video posted on Youtube.
Thus, it is shown that the PBL course provided the students with the experience of resolving real, complex problems taken from the business context (Hmelo-Silver, 2004;Duch, Groh and Allen, 2001;Boud & Feletti, 2003;Hansen, 2006). Thus, this study aims to investigate whether the problems identified, analyzed, discussed and solved in this course can be considered good PBL problems. For this purpose, scores were developed and attributed according to the 11 characteristics presented by Sockalingam and Schmidt (2011), based on the rubrics listed in Tables 1 and 2.

Student
In this educational environment, as mentioned previously, the student is the center of the teaching-learning process. As shown in Table 4, in terms of generation, most students (65%) are between 20 and 25 years old (Table 3). The students worked in small teams of three to seven members, which proposed to cover the literature while assuming responsibility for their own learning as, throughout the course, no lectures on the management accounting issues applied to each problem took place.
The students already had prior knowledge that was developed via interdisciplinarity, building on material from previous years, such as budgeting and planning, cost accounting, and controlling, among others. It should be highlighted that background knowledge also comes from the life experience gained in each student's professional practice, as shown in Table 4. Thus, choosing a problem from a real-life business setting was possible given the profile of the students.
Broadening the benefits of PBL: a "good" problem The students worked together in the same group throughout the course. Stanley and Marsden (2012) emphasize that this is the way to guarantee that the students develop teamwork responsibly within the teaching-learning process. Moreover, by working with problems, students constitute part of their professional life experience and of the context they are immersed in. Thus, this study is presented in accordance with Stanley and Marsden (2012), who emphasize the importance of teamwork in order to solve non-structured accounting problems that are similar to those found in professional practice.
Each group of students, as well as formatting the problem, searched for a solution to a problem that explores a specific area of study, with solutions that are potentially applicable to their contexts of origin. Students were responsible for interviewing key people in organizations and applying tools and scientific research resources, while endeavoring to solve the problem.

Teachers
As already mentioned, teachers assume the role of facilitators in an environment of uncertainty, considering that the students' freedom of choice generates this consequence. The learning in PBL takes place via reflection, dialogue and the exchange of experience between the teacher and student, in which both share the process of constructing knowledge that is embedded in the problem (Decker & Bouhuijs, 2009).
It was observed that the course professor and tutors are responsible for encouraging the participation of all team members in the tutorial sessions, maintaining the focus of the group on the solution to the problem and on the learning objectives listed, evaluating performance, verifying the students' understanding, and certifying that the group achieves the learning objectives proposed in the course. That is, it is extremely important for the teacher to know how to evaluate whether the problem chosen by the students and used in the course promotes the utility characteristics proposed by Sockalingam and Schmidt (2011) and to what extent. Thus, he or she can better guide the groups in constructing the problem and achieving the objectives set out in the pedagogical program, both for this course and the accounting sciences course. Fábio Frezatti, Daniel Magalhâes Mucci, Daiana Bragueto Martins The professor assisted the students in the problem solving process, especially by providing clarification when erroneous concepts regarding management accounting issues were addressed in the problems, and also with literature recommendations that assisted the groups to solve their respective problems. In other words, the professor's role was quite evident in the tutorial sessions.
In accordance with Ribeiro (2008), the teacher, in interacting with the work teams, sought to understand how the students were constructing their own knowledge, while synthesizing together with the teams the knowledge acquired in each tutorial session. Moreover, the tutorial sessions set out to identify and work with the groups that were struggling most, so that they could reach the same stage most of the groups had achieved during the various phases of the course, in order to have equal participation from all of the groups.

Action research
The methodology that permitted the development of the activities during the semester was "Action Research" (Coughlan & Coughlan, 2002) as, although the set of activities was envisaged and defined in advance, it required well specified stages relating with the students. It is understood as a "work in progress" (Wright, Smith, & Duncan, 2011) and, as advocated by Mettetal (2001, p. 108), in the classroom " [...] consists of systematic investigation regarding what works in the classroom, with the aim of improving the students' learning. " Additionally, Wright, Smith and Duncan (2011) understand that, regarding the technique being applied in studies that use PBL, it is satisfying to recognize that people learn in different ways, which allows the students to make choices on the direction of their own learning.
The "action research" cycles used in the study are: (i) understanding the study context, (ii) obtaining the data on the stakeholders involved in the process (students), (iii) feedback for the stakeholders (students) regarding the data obtained, (iv) analysis of the data, (v) plan of action (vi) implementation and (vii) evaluation. The continuity is based on the fact that, at the end of the cycle, a new one begins.

Development, Analysis and Discussion
The course was taught once a year and the fifth version took place in the first semester of 2015. Given how the course has evolved, the last two versions, that is, 2014 and 2015, will be analyzed. In total, there are 17 cases that address very different issues.
Eleven areas were examined separately, in two different ways. The characteristic elements were considered and scores were attributed in accordance with Table 1. There was no concern with weighting the elements in the total calculation but, qualitatively, more attention was given to familiarity and difficulty. For the total, the percentage points obtained in relation to the potential number of points was calculated and, the higher the points obtained, the higher the adherence of the problem to the characteristics. Regarding the intrinsic characteristics, the main aspects considered were: • Familiarity: this was addressed in two different ways: (i) a diagnostic questionnaire applied before the course began and (ii) identification of the issues that could underlie the problem the group would like to address. This attitude involved two elements, which are the potential for clarity and the link with the institutional objectives. It was observed that some problems derived from the content of courses the students had previously studied. In some cases, the problems were addressed, but not with the intensity that they would require. Issues related to psychology, for example, were the most demanded among the cases with reference points that went beyond accounting.
Broadening the benefits of PBL: a "good" problem • Format: the size and the additional elements to the text, such as illustrations and graphs, were considered and encouraged. In the case analyses, this aspect was reasonably homogeneous, as no group was motivated to present anything other than construction via an essay in the structuring of the problem. Regarding the size of the problem description, no great disparity was perceived that led to very long or very short descriptions. What was evident was that some essays were clearer, others more repetitive, and others more or less objective with regard to what would be relevant to the problem. • Clarity: leading on from familiarity and format, the alignment of hypotheses with the problem and with the conceptual content that would be needed to develop and solve the problem was considered. Both clarity and difficulty were legitimized by presenting the problem to the whole class, accepting that the problems would be clear and would contain similar difficulties. • Difficulty: multidisciplinarity, size of institution and demand for references were considered.
The big problem in this area was the overlapping of a complex subject with size of organization, which made the problem complexity relatively high in some cases. The opposite occurred, that is, problems with simple demands in terms of knowledge and applied in small company environments, which made the difficulty low. • Relevance: this basically proposes perceiving who would benefit from the solution to the problem. Problems that benefitted only part of the population in a company would be less relevant than something that could affect the entire company or a region for example. Thus, this area is connected to size and links the element to be researched beyond the entity.
After analyzing the indicated areas, it is expected that the rest of the items be achieved, given the causal relationship that exists between them. Thus, the other six elements in the model can only be completely analyzed after the end of the course. For some of them, a specifically collected variable is available, while the analysis of others depends on a proxy.
• Linking the problem with the learning objectives. This was previously envisaged via familiarity and clarity. As a follow-up, after the end of the course, the extent to which the problem was linked with the learning objectives was evaluated. • Problem arouses interest. Perceived in the group discussions, this can be followed up by attendance at meetings and student effort. In some groups, class attendance became a problem to be addressed, but was not generalized. • Stimulates critical analysis. The existence of more than one alternative solution to a problem or the perception that the problem initially identified addressed simple reasoning. These discussions occurred during the tutorial sessions and this can be shown by means of follow-up and, in some cases, by the minutes. • Problem promotes self-learning. The search for solutions involves references, contacts and interviews at companies and discussions with the professor and tutors. The students' initiative is perceived in this sense and the minutes partially reflect this element. • Stimulates elaboration. The development of the problem and discussions indicates that this area is being achieved. Drop-out or non attendance of groups shows the opposite. • Promotes group work. Working in groups is visible when it takes place in the classroom. The minutes should report meetings and contributions, but this support cannot always be considered reliable.
After the analysis, the problems were compiled in a comparative way as shown in Table 5. The attribution of scores, following rubrics 1 and 2, permits comparative observation. The evaluation can be perceived by the relationship between evaluations of the intrinsic characteristics and of the utility characteristics. A number of cut-off points were established and attention is drawn to four possibilities: • Problems with high intrinsic quality and high utility quality (AA) There were seven problems and, in general, issues commonly found in business areas with well established conceptual requirements. A lesson for teachers of future groups is that the effort to reach the five intrinsic characteristics should continue to be valued and discussed with the students.

• Problems with high intrinsic quality and low utility quality (AB)
There were three groups. Investigating the causes of the characteristics' performance reveals that they seem to be more linked to the profile of the group members. Teachers of future groups should consider whether the composition of teams could not be different, with greater crises to establish the problem, which might enhance the motivation and involvement.

• Problems with low intrinsic quality and high utility quality (BA)
There were two groups and the occurrence and they brought very different issues for situations in which the available literature was not readily usable in a multidisciplinary way. The high result in terms of utility characteristics derives from the fact that the group leader assumed a more aggressive posture, "carrying the weight" of the project. As a lesson for teachers of future groups, questions that are very complex and applied to relatively small institutions could be reflected upon.

• Problems with low intrinsic quality and low utility quality (BB)
There were five groups and, in general, the focus of attention was small companies, which would not be a problem, with challenges not perceived as very significant. The groups were very dependent on the leaders and, in their absence, stagnated. As a lesson for teachers of future groups: greater attention to the combination of dependency on the leadership and the challenge.
Broadening the benefits of PBL: a "good" problem

Concluding Remarks
The purpose of this study was to discuss the intrinsic and utility design characteristics of good PBL problems in a management accounting course that applies the PBL approach. This study provides some guidelines about how to evaluate good PBL problems students proposed in their teams, based on their experiences and contexts. As this environment is not teacher-controlled, it is important that the professor has tools to evaluate how the proposed problems can be changed, arranged, or developed to fit the learning goals of a PBL course. These discussions are relevant as the problem is a fundamental issue for the PBL approach.
The extension of PBL's applicability in areas of business provides a relevant opportunity to look at the approach and attends a profile of students that have the potential to advance in their learning beyond the solution to the case in itself. It is not proposed that this approach be extended and generalized indiscriminately for any course and moment in a course, but, it is an alternative for similar contexts, where the students have more opportunity for contact with a given organizational reality, have academic maturity acquired over time, as well as knowledge in the course in question. Moreover, from the teachers' point of view, the approach requires professionals who want to take risks, which can be supported by the institu- Fábio Frezatti, Daniel Magalhâes Mucci, Daiana Bragueto Martins tions where they carry out their work. It is very important to understand and balance active learning tools with more traditional approaches and the composition of courses undergoes constant adjustments among the various pedagogical project elements of each institution.
The opportunity arises as a result of the context of having these elements and the organization of the courses available. In any case, the proposal developed in this study considers a displacement of risk from the students' viewpoint, in having the freedom to construct their problem and, from this perspective, consolidate their learning. They assume greater risk, being unfamiliar with the method, do not always have a vision of what they can do in terms of constructing a solution, and they still have to decide which type of problem they will be dedicating an important part of their time to during the semester. What is more, all of the elements are addressed collectively, in a small group where abilities and attitudes are encouraged. Even though all of this is similar to the workplace, it is not known exactly to what extent it is similar, and in what depth this will affect their performance evaluation. It is in this point that the role of the teacher emerges, being decisive for the students' success, not only in the short term in the course evaluation, but also as a tool for incorporating lessons into a modus operandi that proves to be individual due to the context of what takes place.
The teacher needs to have the maturity to balance his/her involvement with the students in order to avoid solving all of the students' problems, but should also have the sensitivity to distance him or herself at moments when this is the only possibility for the student to experience the lesson in the context and to an individual extent. Moreover, the risk assumed by the teacher is much greater than what he/she is accustomed to in situations of lectures and exercises and cases in which he/she already has the answer. With this approach, the answers are constructed together with the students, in their time. On the other hand, the lessons learned by the teacher are considerable as he or she is challenged and needs to have answers for elements the teacher would not necessarily value at first. This is important and not necessarily seen as positive. In any event, the proposal does not need to be radical and the level of freedom can be that which the context can use in an appropriate way.
Linking problem, students, and teachers, the intrinsic and utility characteristic tools cause an important impact in the process and its results in making dialogue between teachers, students, and their groups viable, in the sense of orientation to develop the course. In this sense, separation of the model into its intrinsic and utility characteristics (Sockalingam & Schmidt (2011) has practical effects in terms of validation of the problems and their corresponding validation results, not only in a "yes" or "no" to the problems, but also organization and guidance for improvements.
After all, more than having a tool that allows the students to learn in a more context-sensitive and practical way, the aim is to have an approach at their disposal that is long-lasting and effective in their lives, and which will be perceived at appropriate times in the course of their individual careers.