Voorhees Public School District
Voorhees, New Jersey
May 15, 1997
MEMBERS OF COMMITTEE PRESENT:
Assemblyman John A. Rocco, Chairman
Assemblyman Carmine DeSopo
Assemblyman Raul "Rudy" Garcia
Darby Cannon III Christine Costigan Jason Teele
Office of Legislative Services Assembly Majority Assembly Democratic
Committee Aide Committee Aide Committee Aide
(Internet edition 1997)
ASSEMBLYMAN JOHN A. ROCCO (Chairman): The hearing will come to order.
Welcome, everyone. We're going to hear testimony and get input in terms of the fourth-grade assessment. There seems to be a great deal of concern, to say the least, in regard to the test itself, especially the speaking portion of the test for the fourth-grade students. We will try to get as much information as possible, get some of the strategies as designed by the Department, and see if there are ways we can improve the process and, at the same time, resolve some of the issues, to say the least.
We have Rudy Garcia with us. Rudy came all the way down from North Jersey.
Rudy, is there anything that you would like to say at this time as a member of our Ed Committee?
ASSEMBLYMAN GARCIA: No, thank you, John. It's a pleasure to be down here in Voorhees, and I look forward to hearing the testimony. I know some of the educators up from Hudson County and Bergen County -- the areas I represent -- have expressed concern, especially, as John said, over the speaking portion and the types of difficulties that some of the school districts may face with it.
We're looking forward to hearing testimony from all of you today.
ASSEMBLYMAN ROCCO: Rudy did come a long way, and we really do appreciate that.
Other members of the Committee -- we may have a few coming in later; however, most of them are from North Jersey, and we understand the problem that they may have since we did call this hearing rather quickly.
Before we get started, I'll indicate my bias very clearly. I have always said to the Department in this administration, in the Florio administration, in the Byrne administration, and the Kean administration, the Department always presents a problem when it gets into the assessment end of things. They are just not equipped, nor do they have the technical knowledge to put together tests. So I am coming in with a bias.
Those who are speaking on behalf of the Department, we'll listen politely. But, at the same time, all of you already know my bias, so let's just start in and hear from others, because I think it's important to get this input. So let's start. Who from the Department--
Mariann, do you want to start, introduce the members of the Department, and have them make their presentations so we have a base from which to position ourselves?
M A R I A N N R H O D E S: Mr. Chairman, and members of the Committee, this afternoon we have presenting testimony from the Department: Jay Doolan, who is our Director of Standards and Test Assessment, and Gerry DeMauro, who is the Director of Assessment, to respond to any technical questions that you may have, as well.
J A Y D O O L A N: Good afternoon, Assemblyman Rocco, and other members of the Assembly Education Committee. It's a pleasure to appear before you this afternoon to discuss New Jersey's efforts with respect to implementation of the core curriculum content standards and, in particular, the field-testing of the new Elementary School Proficiency Assessment -- ESPA, for short.
After an unprecedented public consensus-building process, the New Jersey State Board of Education adopted a rigorous set of core curriculum content standards in seven academic and five workplace readiness areas a little more than a year ago. These standards describe what New Jersey citizens believe all students should know and be able to do by the time they graduate from public education.
When Christine Todd Whitman assumed the office of Governor, she underscored the importance of arriving at this common definition and recognized, as well, the need to provide the public with a comprehensive accountability system. Governor Whitman played an instrumental role in the development of the standards and invited the public to engage in the discussion about the then-emerging standards.
In January 1995, Governor Whitman emphasized the priority she placed on the standards and an aligned assessment system, which measures student progress in meeting these standards. She proposed an expanded assessment program which would consist of a revised Eighth-Grade Early Warning Test and a Grade Eleven High School Proficiency Test, as well as a new Elementary Assessment which would permit instructional intervention when it is most likely to have an impact.
In its 1994 decision on school funding, the Supreme Court of New Jersey, likewise, emphasized the crucial need for an accountability system which provides useful information regarding how the students of this State are performing throughout their educational experience.
Shortly after Governor Whitman called for the establishment of an assessment at the fourth-grade level, test development activities commenced under a contract between the Department and National Computer Systems, a nationally recognized testing corporation. NCS develops the testing programs using New Jersey educators and Department expertise to review the test items and the administration and scoring systems.
An ambitious implementation schedule for incorporation of the new core curriculum content standards into the fourth-grade examination, as well as expansion and refinement of the existing Eighth-Grade Early Warning Test and the Eleventh-Grade High School Proficiency Test, identifies specific subject areas which will be phased into the statewide Assessment Program over the next five years. New subject areas will be field-tested at the fourth-grade level one year in advance of their incorporation into the eighth-grade and eleventh-grade examinations.
By the Year 2002, all of the seven academic and five workplace readiness standards will be fully integrated into the statewide Assessment Program. Thus, by the Year 2006, students graduating from New Jersey's high schools will have to demonstrate proficiency in all of the content standards.
During the development of the standards and after their adoption in May 1996, there was widespread dissemination of these expectations to districts and schools. District administrators, school staff, and parents are aware of these new requirements for all of our students. Many districts are already aligning their curricula and providing training to their teaching staff.
Prior to the incorporation of particular test items into a formal test, it is common practice among testing experts to field-test items to assure their reliability and validity. Field-testing of certain sections of the new fourth-grade examination will take place next week during the time period of May 19 through June 6. Those subject areas which will be field-tested include the following: reading, writing, speaking, math, and science.
It is worthwhile to note at this time that the citizens of New Jersey greatly expanded the definition of literacy in the language arts. Prior to the adoption of the core curriculum content standards, the statewide Assessment Program at grades eight and eleven measured only student proficiency in reading and writing; however, with the adoption of the new language arts literacy standards, five separate components were identified as essential for proficiency in the language arts. These five component parts include the traditional reading and writing and, in addition, speaking, listening, and viewing.
In preparation for the field-testing of the speaking aspect of the Fourth-Grade Test, much advance preparation was necessary. As part of the content domain and ultimate test specifications for the grade-four examination, as defined by the test contractor with New Jersey educators, all districts with fourth grades were able to familiarize themselves with the ways in which the examination would assess speaking and, in particular, would be aligned with the standards in that area. These materials were received by local districts by early April.
Further, because the speaking component of the Fourth-Grade Test was determined by the contractor, in concert with test development committees of New Jersey educators, to be assessed at the local level, an ambitious plan for preparation of those who would be doing the assessing was put in place.
First, a two-hour video was developed and broadcast on New Jersey Network on April 3 and was rebroadcast on May 7. All schools in New Jersey with fourth grades received a copy of this video so that teachers will have had about six to seven weeks to replay the training session or any part of the training session at their convenience. The Department received many accolades for the video. New Jersey educators, together with child actors, provided example after example of how the assessment of speaking proficiency could occur in an objective and nonthreatening fashion.
Additionally, extensive training of local test coordinators, as well as fourth-grade teachers has been going on throughout the months of April and May. In total, 13 training sessions for test coordinators and 10 training sessions for speaking assessment will have been completed prior to the field-testing of this component of the examination. These sessions were designed to accommodate up to about 3280 teachers involved in the speaking component and 1910 test coordinators.
It is important to note that this field-testing will provide all fourth grades with useful information regarding student performance. The test, designed to be diagnostic and indicative of how fourth-graders will ultimately perform on the Eighth-Grade Test and then later on the Eleventh-Grade Test, will not this year yield individual student scores. Instead, information which may be useful for diagnostic and curricular purposes at the district level will be available upon request.
It's also essential to note that next year, while cluster scores will be provided on performance in the areas of reading, writing, mathematics, and science, the speaking assessment will stand alone from the scoring technique. Instead, a separate indication of speaking proficiency will be provided and will not be folded into the language arts score.
I shall also note that there are procedures in place to excuse children for whom speaking causes undue stress. This practice is available for speaking, as it is for all aspects of this test and others.
I think I'll end there, and thank you for the opportunity to address this hearing. Dr. DeMauro and I will be happy to answer any questions that you or others on the Committee may have.
ASSEMBLYMAN ROCCO: Thank you, Jay.
There are a number, and I'll try not to dominate too much here. The issues that have been brought to me, I think, I can present to you, and we'll have many teachers and Superintendents who may have comments. But I think--
Let me, in a broad sense, say that the issues that are of concern to me are the ability to deal with basic statistical information, such as validity and reliability, and the speaking portion. I think it will be an impossibility to get either validity or reliability in that process, nor do I think you can use whatever information you receive in terms of judging one district against another within the State. I think that's a very significant problem.
It's not an objective evaluation. It's a very subjective evaluation, therefore, you're going to get all kinds of-- Fifteen people could look at the same individual and come up with 15 different scores. So even with the matrix, etc., it just seems to me that you're never going to be able to get any validity or reliability in that speaking portion. And basically, most of the complaints I've received -- and there maybe others that will come forth -- deal with that particular portion of the test.
The others deal with time commitments, the norming of the tests, and, of course, having to bring in other personnel into the classroom to deal with the evaluation of the students, since there are two evaluators in the process.
But, again, I don't want to dominate on that issue. I'm going to let others-- I'll dominate later.
Rudy, do you want to -- do you have anything you want to say?
ASSEMBLYMAN GARCIA: Yes.
ASSEMBLYMAN ROCCO: And Carmine DeSopo -- Assemblyman DeSopo -- has joined us, along with Dr. Harris from Rowan University, our consultant.
ASSEMBLYMAN GARCIA: Basically, just one question. Are the 1997 scores reported on a student basis or just on a district basis?
MR. DOOLAN: Neither.
ASSEMBLYMAN GARCIA: Neither?
MR. DOOLAN: They won't be reported at all, essentially. They will be available to districts in some way, based upon district interest and request.
ASSEMBLYMAN GARCIA: When will they start being analyzed and distributed to all the districts and compared with all the districts?
G E R A L D E. D e M A U R O, Ph.D.: 1998. In 1998, the scores will be -- the results of 1998 operational test will be used for standards' setting. So it's not normed at all. There are no norms here being developed at all, just criteria and reference.
ASSEMBLYMAN GARCIA: So how are the comparisons going to be made, or how is this information going to be useful in assessing how far along a school district has come, or how will that be measured?
DR. DeMAURO: The test is derived from the contents standards, the test specifications.
ASSEMBLYMAN GARCIA: And we're just talking-- If we could focus on the speaking portion, then.
DR. DeMAURO: Well, speaking, that isn't part of the comparison.
ASSEMBLYMAN GARCIA: So it won't be part of the comparison at all.
DR. DeMAURO: No. The standards call for and evaluate a speaking experience, and so we developed a rubric that teachers can use over the course of the school year, whenever they want, and we have a time for them to do the assessment. So it's not being used at all to compare districts.
ASSEMBLYMAN ROCCO: So you're not-- Let me just follow-up on--
Rudy, if I may just jump on your question there.
So you're not going to use these statewide on the speaking portion?
DR. DeMAURO: No. It's a local assessment.
ASSEMBLYMAN ROCCO: They're going to be assessed within the district and remain within the district.
DR. DeMAURO: Yes, sir.
MR. DOOLAN: Yes. It's a local assessment.
UNIDENTIFIED SPEAKERS FROM AUDIENCE: (indiscernible)
ASSEMBLYMAN ROCCO: No, I think you're getting a little disagreement behind you there.
ASSEMBLYMAN GARCIA: I think the question from the audience, from their reaction, may be: What's the purpose of this then?
DR. DeMAURO: It's in the standards. The standards requiring that speaking be developed and that the children have an evaluated speaking experience before an audience. So, because they're part of language arts literacy, we used the specialists in the State in language arts literacy to develop a rubric. A large of part of our objective was met as soon as the rubric hit the hands of the teachers, because they can use that to evaluate speaking.
Now, what the State is doing as part of the field test is, we have several experiments going at the same time to see what kinds of teachers are best for the kids; how to get the best kind of reliability in terms of the combination of scores with respect to the rubric; how to implement it. Administration -- we let districts decide on their own way of implementing it, which ways worked best to garner validity and reliability.
But it's a local assessment. It's a local assessment.
ASSEMBLYMAN ROCCO: Does it stay within the district, or is it used to measure district against district statewide?
DR. DeMAURO: No.
UNIDENTIFIED SPEAKERS FROM AUDIENCE: Why are we mailing it back again? Why are we mailing back? Why do we have to send it back?
ASSEMBLYMAN ROCCO: All right. There are questions about--
DR. DeMAURO: We're studying the way it's implemented.
ASSEMBLYMAN ROCCO: All right. You're going to study the way it's implemented. All right.
I guess the other part of that question -- and, Rudy, you may have more and jump right in--
ASSEMBLYMAN GARCIA: I will.
ASSEMBLYMAN ROCCO: Since this is Rudy's time, anyhow--
On that same issue, it seems to me that -- and, again, we're not here for roasting, we're just here to get information and to deal with this issue -- there is a lot of either misinformation out there or, even if it were to remain within the district, how can you get true validity or reliability on such a subjective evaluation?
DR. DeMAURO: Validity-- Which do you want me to respond to, validity or reliability?
ASSEMBLYMAN ROCCO: Both. I think both are not there.
MR. DOOLAN: We'll start with validity first.
DR. DeMAURO: I'll take validity. Validity, because we have-- It's different with this test than it was with any test in the past because we have published content standards, and the objective was to develop an instrument that was sensitive to the acquisition of the K-to-four standards. So until we have empirical data against which to test, the validity is really construct validity in the sense that we start with the standards and the CPIs. We have committees of specialists--
ASSEMBLYMAN ROCCO: Okay. I'm sorry, let me interrupt you. You're talking about the fourth grade in general, right? I think--
DR. DeMAURO: Well, speaking, as well.
ASSEMBLYMAN ROCCO: I'm focusing on the speaking portion of it, yes.
DR. DeMAURO: Validity is the extent to which you can support the score-based inference made on a measure. That is the definition of validity. And we've done that in several ways with several Committees in developing the rubric.
Now, the reliability issue-- If you don't have reliable application of the rubric, then validity is compromised and the extent to which that's true, district by district, presents validity as a variable.
ASSEMBLYMAN ROCCO: And consistent, you need consistency.
DR. DeMAURO: Absolutely. That's right. And especially consistency within the classroom, which is more important across classrooms because the scores are being made on students. Across classrooms there's no basis for comparison--
Now, reliability actually has two components, especially in an instrument like this. Reliability is the agreement between the observed score and the true underlying score or the true ability of the child. The first component, we estimate through-- The second component is interrating reliability. We need two people. And you need-- To garner reliability at all, you need two people.
We didn't want to put two strangers in the classroom. We wanted the child to speak in front of his most comfortable or her most comfortable audience, which included the classroom teacher; although, we've allowed districts the option of, for example, bilingual children to speak in front of the bilingual teacher, whatever teacher they're most comfortable with. The second scorer is to test whether or not the scorer outside -- who is not the most familiar to the student, can also give the same kind of reliable score. If we find that that correlation is very high, then there is probably no need to use two scorers, but that's the second aspect of the validity -- interrated--
The first aspect of validity is empirical. That is that if this contributes to a score that has several components to it, then we can analyze the contribution that it makes to that overall score. In this case, it's not, but we can still analyze it as though it were. We can do an empirical analysis. It's an empirical question. It's not one that somebody can say beforehand it exists or it doesn't exist or the extent to which it exists.
ASSEMBLYMAN ROCCO: Well, I mean, we know that we can take an essay and throw it down the steps and put A, B, C on various steps-- And with an essay-- And you can get 10 different evaluators on an essay and get 10 different scores. I think in this case, from a speaking perspective, you can -- you'll get 10 people with 10 different areas of the rubric making their determinations. I don't see reliability coming out of this--
DR. DeMAURO: It's not been our experience.
ASSEMBLYMAN ROCCO: --number one. Number two, when your dealing on a one-to-one situation with the speaking, it's so time consuming within the classroom environment -- and I've taught fourth grade, so I'm not speaking from what I've heard, I taught-- You need to have time to do this, number one, which means you're taking time away from other subject areas, which is already limited, to say the least. Then, you have to bring in additional personnel to do the evaluation. Did anyone ever stop to think "Well, this will be costly or time consuming?" Has that been taken into consideration?
DR. DeMAURO: Of course.
ASSEMBLYMAN ROCCO: And what did you determine?
DR. DeMAURO: We're trying to implement the content standards. We're trying to do it without using the costly commercial instruments that don't even test in the subject area that the content standards address.
ASSEMBLYMAN ROCCO: Well, I've always been at odds over the Department. Why not just use the nationally normed test because they're out there and commercial and already normed. And I don't think Jersey is all that different, but I don't go into that battle with you guys--
But the other -- on this particular-- I mean, this is so far afield from true validity and reliability and consistency that this whole issue is one -- and the other issue being time consumption and the cost of additional personnel, what do you say to districts? Hire somebody, bring them in. Who are you talking about bringing in here?
MR. DOOLAN: I think in terms of the speaking, we are mandated to include the speaking because that was an additional requirement in terms of the standards. In deciding, really, how to address the speaking section, we worked with educators across the State, the testing company to develop a process that would be the least intrusive but would also measure the value of speaking that the public, the Legislature, the State Board of Education indicates should be included, because the standards were widely disseminated, there was a lot of input, and speaking was always there.
I don't know to the extent that additional personnel have to be included. In our training programs, we do include lots of activities that the folks in the schools can participate in. I can tell from the whispers in the background that people don't agree with that, and I'm anxious to hear that, as well.
ASSEMBLYMAN ROCCO: Yes. I think it's very important. That's one of the reasons we're having it. Again, I'm not trying to put-- All I'm trying to do here is to show you what problems exist out there, and you're going to hear, I'm sure, in that regard many of the concerns.
So why don't I just kind of see if--
ASSEMBLYMAN GARCIA: Let me just, if I may, John--
ASSEMBLYMAN ROCCO: Go ahead, Rudy.
ASSEMBLYMAN GARCIA: Who's going to evaluate whether this test is effective? Is there an evaluation period? And is the Department of Ed going to do it or the company that devised the test?
DR. DeMAURO: We do-- We actually use both. The company, that is under contract, does studies and helps us design studies that we ask for. But we do lots and lots of studies on the validity of the existing instruments, as will continue with this.
ASSEMBLYMAN GARCIA: Just let me ask you a question. This test, is it going to be compared, or are they taking anything from the national tests that are out there, or is this just strictly devised by the State of New Jersey just to measure content standards?
DR. DeMAURO: Yes.
ASSEMBLYMAN GARCIA: Or is student performance not part of this?
DR. DeMAURO: Student performance is-- The test is broadly diagnostic in the sense that it distinguishes--
ASSEMBLYMAN GARCIA: Because one of the things-- I know in certain districts, people will teach to the test. If this test is strictly based on the content standards, it could be easy -- especially once the test is out there -- for teachers to start implementing a program that will only go to content standards, and then, after that, really what is the effectiveness of this test as a tool to measure student performance?
DR. DeMAURO: The test is actually not a static thing. It's an ever-changing item pool. That's why we need a field test to get it started. And that item pool is part secure and part released. Now, the released items address the specifications. There are samples of the test specifications. The secure items, they don't have access to, and that's an ever-changing pool with some overlap for equating purposes. So they can't really teach to the test as a static, existing thing. That wouldn't be possible.
ASSEMBLYMAN GARCIA: I don't know if I've been with John for too long now, but my question then becomes: Why is New Jersey-- I mean, honestly-- And I know John -- this is his issue -- and he could speak to it better than I can, but really-- If there are all the tests out there that measure -- because the content areas are basically more or less what every other state teaches their children--
You're shaking your head, no. But for -- I'm looking at this, it's mathematics, reading, writing -- those are basic, the three R's. I don't know what else-- Why is the State even involving themselves in this?
DR. DeMAURO: There are seven content areas and workplace readiness-- There are no instruments that test that. The commercial instruments available--
ASSEMBLYMAN GARCIA: So what instruments -- how are you going to test workplace readiness of a fourth-grader?
DR. DeMAURO: Workplace readiness has five components in it, and they are phased in one year at a time.
ASSEMBLYMAN GARCIA: What are they?
DR. DeMAURO: The first is critical thinking. I know the first few, because those are the ones they're most concerned with.
ASSEMBLYMAN GARCIA: Right.
DR. DeMAURO: Safety.
MR. DOOLAN: Technology.
DR. DeMAURO: Technology is second. They're imbedded in the test. The way we measure that is through a profile.
ASSEMBLYMAN GARCIA: If I may, how do you measure technology?
DR. DeMAURO: There are items on the test, in the context of the subject matters, this -- the workplace readiness -- cut across the subject areas that require the use of technology as it's described in the standards.
After critical thinking, there are items in the test that are on higher inferential levels -- problem-solving skills -- and that's part empirical and part judgmental, because we will also see that other item's load on the same attribute.
MR. DOOLAN: And, Assemblyman, I should say that not all of the workplace readiness standards are being assessed as part of this Fourth-Grade Assessment, that we're phasing in the standards over the next several years. This year, we're focusing on critical thinking, and that will be a part of the Fourth-Grade Test.
ASSEMBLYMAN ROCCO: Well, Rudy's a lawyer, and I may be the only fourth-grade Chairperson who has ever Chaired this Committee in the history of the State, but we know what we're talking about here, really. What Rudy's talking about, I think, is the basic math skills, reading skills, language skills, etc. We all know that Californias, the Metropolitans, the Iowas, etc., we can name test after test in which-- There are nationally normed tests out there.
New Jersey, for years, just felt that they had to do their own testing and has spent millions of dollars. Whether it was a Republican administration or a Democratic administration, they just felt that they had to do this testing, for what reason, I still don't know.
The only thing I received was some information that says, basically, that nationally normed tests aren't really nationally normed, which means the test makers are lying. So you can take E.T.S. and basically state that they're not being legitimate with the public, but that's a whole different story.
MR. DOOLAN: Assemblyman, our focus was on implementing the content standards which we feel are high quality and at a high level. In looking and reviewing the commercially available tests and the national tests that are out there, we did not feel that those tests adequately measured the standards as we had developed them in all of their complexity here. That's why the focus has been on accountability and on developing the test -- a statewide test.
ASSEMBLYMAN ROCCO: Well, are you stating then, if we gave the nationally normed test, our kids would score higher than the rest of the states?
MR. DOOLAN: I don't know enough about that.
DR. DeMAURO: No. I mean the logic is then on the State test, and that's true. Also, we don't use the test normatively.
ASSEMBLYMAN ROCCO: What's true? What's true? No one-- If you're saying you have to develop a different test because our skills are so much higher than other skills in other states-- Is that what you just said?
DR. DeMAURO: No, I didn't say that. I said compared to the objectives of the normed tests. I didn't compare them to other states. That may be true, but that's not what you're asking.
ASSEMBLYMAN ROCCO: Yes, it is. What I'm asking--
DR. DeMAURO: I'm sorry. I misunderstood. I thought you were asking for comparison of the New Jersey test to the nationally normed tests.
ASSEMBLYMAN ROCCO: What I'm asking is, if we took a nationally normed test and gave it to New Jersey students -- which this district and every other district does -- why wouldn't that be acceptable for determining whether or not our students have math skills?
DR. DeMAURO: Math skills is a very wide domain. As you just said, there are basic skills.
ASSEMBLYMAN ROCCO: Well, usually in fourth grade, you have the times table, you have-- You know, it's pretty standard stuff.
MR. DOOLAN: In looking at the content standards, as they've been developed in New Jersey, and comparing them to the test specifications of some of the commercially available tests and also the national tests, not all of the skills that we are teaching and expecting our students to learn have been covered. So that we wanted tests that would give us information about the bulk of the skills that we're expecting our school districts and our teachers to teach. And as an accountability part--
ASSEMBLYMAN ROCCO: The bulk of the skills in fourth grade would be addition, subtraction, multiplication, a little division, a little decimals, pretty much--
MR. DOOLAN: I think you'll find in looking at the fourth-grade standards, as we've outlined them, that there are skills that are somewhat higher than that. There are different skills as measured by others--
ASSEMBLYMAN ROCCO: Right. Okay. Thank you.
So therefore, fourth-graders taking the test could, if we gave them a nationally standardized test, score at sixth-grade -- like six one or six two, right? Because then they would have the higher order skills in multiplication, decimals, fractions, etc., right?
DR. DeMAURO: Well, we would be unable to gauge--
ASSEMBLYMAN ROCCO: Are you following me on this or am I just--
DR. DeMAURO: We would be unable to gauge where they would score because the test is aimed at a different content.
ASSEMBLYMAN ROCCO: No. If they had skills above and beyond the fourth-grade level, they would score above and beyond the fourth-grade level.
DR. DeMAURO: On those skills.
ASSEMBLYMAN ROCCO: If you take a fourth-grade test, you have many students who come out with sixth-grade skill levels. Right?
DR. DeMAURO: With respect to the skills. It's not-- The score doesn't have an intrinsic value for the whole of math. The score is with respect to the skills that are being measured.
ASSEMBLYMAN ROCCO: Why doesn't it have an intrinsic value?
DR. DeMAURO: Because they may not measure things like problem-solving skills. When we had the MBS, for example--
ASSEMBLYMAN ROCCO: Well, certainly, there are problems given in nationally normed tests.
DR. DeMAURO: They may be measuring mathematical facts, computational facts of the kind you said. They may not measure things like transitivity seriation, inferential skills, problem solving, problem setting up.
ASSEMBLYMAN ROCCO: Oh, I think they do. I think they do. All right. I won't debate the issue.
Carmine, did you--
Rudy, were you through?
ASSEMBLYMAN GARCIA: Yes. I just wanted to say. I know you probably prepared this testimony before yesterday, but maybe you could also add that now the 1997 Supreme Court has given us the money so we can teach the content standards in the special needs districts. (laughter)
MR. DOOLAN: Thank you for pointing that out. (laughter)
ASSEMBLYMAN GARCIA: I'm glad you are aware of that. (laughter)
ASSEMBLYMAN ROCCO: Carmine.
Rudy wants to get a political hit in there. Go ahead. (laughter)
ASSEMBLYMAN GARCIA: No, no, John. Not at-- This is not directed to the Department of Ed.
ASSEMBLYMAN ROCCO: Carmine.
ASSEMBLYMAN DeSOPO: I better not make any jokes at this time.
But I'm not quite sure whether the horse is out of the barn. I think I understand what they're trying to do, that they're taking a great deal of pride in developing the core curriculum and then developing the standards and developing proficiency or measurements to measure those skills. And that they're unique and something that we're proud of in the State of New Jersey, but at the same time, I don't know if we are realistic in -- when we're doing these things, how we're doing these things--
I think most educators are very proud to have high standards in education and they want to accomplish these things, but I do have the same concern that Chairman Rocco has expressed, primarily in the speaking portion, again, in the area of validity and reliability and diluting our resources -- whether it's necessary. What are we hoping to accomplish? The cost benefit of some of the commitment that we're doing in order to have these things--
I'm not quite sure, based on the kind of student body that we have in the State of New Jersey -- with a tremendous, tremendous amount of youngsters from other countries in the State of New Jersey -- whether it would be beneficial for these people to be put in that kind of situation. I don't know what kind of benefits we're trying to derive from that, or whether it's simply an academic exercise for those people, that it looks good, that it's something that's part of a total package that we can tell the rest of the country that it's a nice, little package -- so I have those concerns -- or whether it's really beneficial that we bring in people to give these tests, especially since they're going to be reported. Again, I have some very strong concern in the administration of those particular tests.
I fully do not understand how you're going to go about doing it and, also, with special ed, since I don't see any precise curriculum -- core curriculum -- that you've developed, that you're trying to measure it. And you're trying to measure-- I do not fully accept the view that every one of those kids should be able to go through those core curriculum. I know that there are going to be some exception, but, at the same time, I don't see the clear vision that you have in that area, and you're talking about a lot of kids in that area.
So I'm extremely concerned about the entire population at risk, whether we -- you know, from the slow learner to special ed kids-- I don't think you have any plans to test the kids all in private schools, also? It is my understanding some are being tested, some are not being tested. So the question is: What are we trying to prove? Where are we going? What are we doing?
Are we better off concentrating on those cognitive areas, as Chairman Rocco said, that we can really measure and really focus on those things and get everybody and say, "Okay, these are the two, three, four areas, essentially, in the next few years, that we're all going to be on board, and we're going to focus on that"? As opposed to diluting our efforts in so many areas, in some of those areas where we're going to have put more resources. I don't think that I've seen at this time a manifestation that there is going to be a cost benefit or a benefit in terms of adjusting programs and curriculums to meet the needs of the kids.
I think this is why Chairman Rocco was asking how will you be able to derive information that will be helpful in adjusting the core curriculum so that it's constantly alive and changing and to be able to evaluate and measure it, when there is so much subjectivity especially with the kind of population we have in the State of New Jersey, also -- not just in the administration, but also the population.
So I have some real concerns and nobody has sat down and said to us, "Here's the way we're going to try to do it. And, yes, we do understand some of these concerns." I have the feeling that maybe a lot of people in the State, also, haven't fully internalized this whole thing, and that maybe this particular hearing will be beneficial for you, also.
ASSEMBLYMAN ROCCO: Yes, Jay, if you and Gerry would just kind of stick with us and hear the testimony, it might give you some new perspectives that you may not have considered previously.
So we thank you.
MR. DOOLAN: We'd be happy to do that.
ASSEMBLYMAN ROCCO: Thank you for your presentation.
MR. DOOLAN: Thank you.
ASSEMBLYMAN ROCCO: Let's have Ray Brosel, the host Superintendent here in Voorhees who was nice enough to have us here.
Ray, did you have any words?
R A Y M O N D J. B R O S E L JR.: (speaking from audience) I have a few, and I'd defer to my assistant who is in charge of (indiscernible) has tried to implement this.
So I am Ray Brosel, Superintendent of Schools for Voorhees, New Jersey.
ASSEMBLYMAN ROCCO: I think they need to pick it up on-- (referring to microphone)
HEARING REPORTER: Yes, thank you, Mr. Chairman.
MR. BROSEL: I can do that from here? (referring to PA microphone)
ASSEMBLYMAN ROCCO: No, I think you have to be picked on the mike there. (indicating recording microphone)
MR. BROSEL: Fine.
We'd like to thank--
ASSEMBLYMAN ROCCO: By the way, everyone should know a transcript is being made of the hearing here today.
MR. BROSEL: And after all that, should I start again? (laughter) (affirmative response)
Thank you. I'm Ray Brosel, Superintendent of Schools of Voorhees. We thank you very much for taking the time to have a hearing, and I thank Assemblyman Garcia for coming. I've seen him in Trenton many times, and this is a distance for him to come. I thank you.
Carmine, thank you for taking the time to come.
And, Assemblyman Rocco, we certainly appreciate it.
The concern is the misinformation we received, the cost to the local districts, and the complaints that you're receiving. For instance, we have about 19 fourth-grade classes. No matter how you look at it, that's going to require me to disrupt 38 people in some fashion to do the speaking part of the program. I would certainly ask that you ask, what exactly is the speaking portion? It's my understanding it's a two-minute presentation which the young person has about an hour to prepare for.
I'd also ask you to take a look at the amount of time we're spending for testing. It's from the 19th to the 6th of June. We have a major concern for the amount of time that that--
ASSEMBLYMAN ROCCO: The 19th of May to--
MR. BROSEL: To the 6th of June, is the window to do it.
ASSEMBLYMAN ROCCO: That's a term of three weeks.
MR. BROSEL: Well, that's a window. I'm sure you can do it in less time in if you have 20 youngsters. If you have 400 youngsters, it might take the three weeks. If you have 1000 youngsters, it could take a while.
We have a concern about the time, the cost and, at this point, I'd like to ask Mr. Pollock if he would go over the preparations that we have to make in order to administer the test.
We'd also like to get a final answer to the question-- We also provided our youngsters the opportunity to take the CAT test, and I'm sure a lot of districts have implemented and administered a standardized test some time this spring. If this is to replace that in fourth grade, we would certainly like that direction at some time in the future.
ASSEMBLYMAN ROCCO: Well, in fact, why don't we try to get--
Is this meant to replace the nationally normed tests at some point?
MR. DOOLAN: (speaking from audience) Yes, it is. And that will become official next year when the fourth-grade test is not a field test. It will be given the actual administration.
ASSEMBLYMAN ROCCO: Could you pick that up? (speaking to hearing reporter)
HEARING REPORTER: Sort of.
MR. BROSEL: We await the memo.
ASSEMBLYMAN ROCCO: Thank you. (laughter)
MR. DOOLAN: You'll get it.
ASSEMBLYMAN ROCCO: Mr. Pollock.
A L E X W. P O L L O C K: Thank you.
My name is Alex Pollock. I'm Assistant Superintendent of Schools, and, unfortunately, I have been nominated as Test Coordinator for our district. (laughter) I guess when you're Superintendent you can give someone else that choice.
There are several concerns that we have, and the first I'd like to address is the core standards. Actually, we're in favor of them. I think the standards are appropriate. It makes our job a little easier in the sense that our community is very mobile. We have kids in and out constantly and certainly it would be helpful to us if kids had similar experiences across the State when they arrive in Voorhees Township. So from our perspective, we're certainly in favor of the core standards.
Where the dilemma arrives is the assessment of those core standards. Earlier this month, we gave CAT tests to students. We spent seven days giving a standardized test to these fourth-graders. Next week, we start giving them this test (indicating) the Fourth-Grade Assessment, which will take us four days just for the reading, the math, and the science, and depending on the individual school in our district, one to three days to give the speech part of the test. So as you can see, a good chunk of May has been basically destroyed as far as instruction is concerned.
Along those lines, we will have spent approximately $3500 for training purposes for this test. The main reason for that is the requirement of having two people assessing the speech section of this test. Maybe some people really enjoyed watching the video for two hours. Our teachers didn't. I didn't.
ASSEMBLYMAN ROCCO: I didn't. I saw it.
MR. POLLOCK: And as far as being helpful, I don't believe it was at all. The rubric does provide some assistance, but I can't believe for one minute there is going to be any reliability from one classroom to the next using that rubric or assessing the language part or the speech part of that test.
In some respects, this is water over the dam. The fourth-grade test is going to take place, and we'll do the best we can with it. My concern is more for the future. We're assessing this year three parts of that test, and it's not even the complete three parts. What happens when we have to assess all seven parts of that test, as well as the cross standards? Are we going to take 20 days, at that point in time, to assess all these standards? How many more people are we going to have to have in the classroom? How many more people are we going to have to train?
This is not like giving the high school test or the Eighth-Grade Test. Most districts have one high school. Most districts have all of their eighth-graders in one place. We have four elementary schools, 20 fourth-grade classes, so we need test coordinators in each one of those buildings. We need someone running up and down the hall to maintain the security, which almost becomes an impossibility when you're dealing with so many people shifting tests across so many spans of time, so many hands touching them, and that becomes a concern because we certainly made our staff aware about the security issues related to this test and what happens if those items do become available to the public or to other members of the staff.
Moving along, if you assume that in the future we're going to assess all of those areas in seventh grade, time becomes a major issue. Now, let's move along to the eighth grade. Our district is a K-to-eighth district. I assume what takes place in fourth grade, eventually, takes place at the eighth-grade level, as well. Where do we find the time to give eighth-graders a test that I'm going to estimate is going to take 15 to 20 days based -- if three sections of the test take about 5?
Why would an eighth-grader, going on to a regional high school, put any time in on a test like this at the end of May? I think if you have been an eighth-grade teacher or in an eighth-grade classroom at this time of year, you'll find that eighth-graders' minds are long gone at this point in time. They're looking toward their freshman year, not taking some type of test.
So my fear is for the future, not for this fourth-grade test. If we can have some mechanism in place that cuts down the time that may be needed to give the test or, perhaps, spread it out across some grades. For example, when we add the social studies and the other parts of the fourth-grade test, maybe it really should become a fourth-grade test/fifth grade test with parts in two different grade levels, and maybe that may need to be true for the eighth-grade test, as well.
It's too much for the month of May to give this amount of testing and expect to get any validity out of it, especially the further you go up the grades, especially if your students are going to a regional school district and they're leaving you and they couldn't care less about a testing situation. So I hope attention is given in the near future to what happens next year and the year after with the amount of time required for this test not only at the fourth-grade level, but at the eighth-grade level, as well.
ASSEMBLYMAN ROCCO: Mr. Pollock, the issue of bringing in additional personnel-- How do you bring that second person in?
MR. POLLOCK: We've taken--
ASSEMBLYMAN ROCCO: I assume it's the classroom teacher and someone else. Correct?
MR. POLLOCK: Yes, it is. We've taken reading specialists from our building. We have reassigned basic skills people. We have reassigned any certified elementary teacher that we have available within that building to assist in the administration of the speech part of the test.
ASSEMBLYMAN ROCCO: So you take them away from their existing responsibility in order to try to meet the compliance for this test?
MR. POLLOCK: We're taking them away from their existing responsibility and hiring substitutes to replace them, because we believe there is a need for them to be where they were hired.
ASSEMBLYMAN ROCCO: That, opposed to bringing substitutes to do the evaluation, who really don't know the students at all?
MR. POLLOCK: Correct.
ASSEMBLYMAN ROCCO: So that's where the cost factor comes in, Jay, if you'll keep that in mind. So other superintendents may feel the same.
Are there any other questions for Mr. Pollock? (no response)
MR. POLLOCK: Thank you.
ASSEMBLYMAN ROCCO: We appreciate it.
Let's go to the prominent alumni of this great district, Michael Johnson -- soon to be President of the NJEA.
Michael, I know you have some teachers with you.
M I C H A E L J O H N S O N: Yes, we do. Thank you.
I'd like to begin by thanking the Committee for holding the hearing; it's extremely important.
I think one of the first questions that I would like to pose is that given the fact that the core standards were just adopted this past spring -- and we're talking about implementing a fourth-grade test next year-- We're talking about testing a group of students at a fourth-grade level, who have only been through one year of the core standards. I would question the validity of what we're going to learn until the Year 2001 as to the impact positively or negatively of the core standards, because the children will not have been through four years of the standards themselves.
As you probably are well aware, the NJEA has a long-standing policy relating to the testing of students. We believe that standardized tests have a place in the curriculum, but any educator will tell you that a standardized test is nothing other than a snapshot of a child's level of expertise in a particular area. That singular omits the panoramic view of the classroom teacher. It ignores the progress the student has made during the school year as a whole and often disregards the state of mind of the student during the administration of that test.
We believe that a more accurate evaluation should include all of the above and must not rely on a sole instrument. From what I've learned about this particular instrument, I assure you that the classroom practitioners across this State are shuddering at the prospect of the potential damage this test can do to their students.
As an elementary teacher, I'm a bit surprised that the Department of Education is supporting a test that will take 11 hours to administer. This is longer than the time that it takes to administer the SAT college entrance exam.
To schedule the pilot test for three weeks at the end of May or the beginning of June is also problematic. Most school districts will have just administered another full battery of Iowa tests or the California Achievement. And while we've heard that this would take the place of the standardized test, will it also take the place of those school districts randomly selected for the NAEP, which is given at the Federal level?
I wonder seriously and how seriously nine-year-olds will view a test at the end of the year. This obviously should be considered when the pilot is completed and this test is scheduled for the future.
There are two specific areas of concern that a number of fourth-grade teachers -- who are currently piloting this test -- have brought to my attention, and I would like to share them with you now:
One, the public speaking component of the test. This component requires fourth-graders to spend one hour reading material and preparing a two-minute speech. Each child must work alone for the prescribed amount of time and may not practice in front of other children. I would like to address those two aspects.
First of all, I have a serious doubt that many fourth-graders have the ability to spend one hour on a task solely unsupervised. Secondly, as an NJEA officer, I gave a great many speeches every year. I can assure you that my least-effective speeches are the ones that I have not practiced. We, I believe, are building a program that is going to create certain failure with this approach.
Teachers involved in this pilot have been told that all children -- all -- will be required to participate in the public speaking component, whether they are limited-English speaking, whether they are auditorially handicapped, these children are to be involved in the public speaking component. That is what our people are being told. This is, to say the least, an extremely frustrating exercise for children in this set of circumstances. How will they be helped by an exercise that will almost certainly mean failure for them?
The second concern that our teachers have with the public speaking component involves the manner in which it must be administered. Two teachers are required to grade each student. Who will be responsible for the second teacher's student while he or she is assisting in the testing of other students?
You've heard the plan outlined that is being utilized in Voorhees. In many school districts of this State, they do not have the ancillary personnel to supplement for the second practitioner. Therefore, you're talking about another classroom teacher. Will that teacher be expected to prepare a day of lesson plans for a substitute to cover their class? Will the school district be expected to absorb the cost? Will other teachers be expects to cover that classroom? Will students be combined into already existing classes? There are far too many unanswered questions regarding just this one area of the test. In fact, there are far too many easily identified problems.
The second major area of concern from our teachers involves the training of those who will administer the test. Now, I'm sure that you would be, at the very least, disturbed to hear that at least one trainer has taken a somewhat defensive attitude when teachers questioned the validity of certain areas of the test.
I also think that you would agree that it's highly inappropriate and unprofessional for concerns raised to be brushed off and for them to be chastised with statements such as, "You know, everybody had the chance to help write this test." Everybody, who? We constantly hear that from the State Department. "You were involved." Not a single teacher who I have spoken to has told me that they were asked for their input on this test.
I think you will also agree that the trainers who have been hired by the State should be ready, should be willing, should be eager and able to hear comments from classroom practitioners who will administer this test and to do it in a professional and supportive fashion. If they themselves were classroom teachers in New Jersey's public schools, I would expect that their evaluations would be far from positive in nature if the evaluators heard such comments being made.
My remarks have only focused on a few of the areas of concern that our teachers have voiced. I'm pleased that some of the practitioners involved in the pilot program are here today to give you a firsthand account of some of the concerns that they have.
I hope that you will encourage the Department to rethink the test. I hope that the timetable for adoption will be reexamined, and I hope that recommendations from those responsible for administering the test will, in fact, be heeded.
Thank you very much.
ASSEMBLYMAN ROCCO: Thank you, Michael.
Did you have teachers who you wanted to--
MR. JOHNSON: Yes, we do. We have several people here from Morris County, in fact.
ASSEMBLYMAN GARCIA: Michael, this test takes 11 hours to administer?
MR. JOHNSON: As we went through the manual and began adding up all of the time that is required for the students to be involved, it is an 11-hour process.
ASSEMBLYMAN GARCIA: That's not only longer than the SATs, it's longer than the LSATs.
MR. JOHNSON: Yes, really. So that itself is rather astounding.
ASSEMBLYMAN ROCCO: Good afternoon. Do you want to just introduce yourself and get started?
D O N A L D L A G O O N: Good afternoon. My name is Donald Lagoon. I'm here with Pat White, Joanne Mein, and Harriet Casper. We're from Morris County, Rockaway Township. We'd like to address some of the concerns that have already been mentioned, and as a preface I'd like to say thank you for having this hearing, because I've already learned more than, perhaps, I've learned in the past.
ASSEMBLYMAN ROCCO: Well, that's good.
MR. LAGOON: So it's an educational experience. As I mentioned, I do teach in Rockaway Township in Morris County. I've been a teacher there for five years. I come to you this afternoon with a background in education, including being chosen as a representative by Rotary International to represent U.S. educators in Japan, a Teacher of the Year in my former district in Minnesota, where I taught for 14 years, a Minnesota Honor Roll Teacher, and a Minnesota Teacher of Excellence. Hopefully, these qualify me to express the concerns my colleagues and I share regarding the Elementary School Proficiency Assessment, which is to be administered in my district starting next week.
It is not my intention to tell you that standardized tests for students in New Jersey should be omitted. Quite the opposite is true. There is a place for standardized tests that measure a wide variety of educational components; however, there are problems regarding how this test has been delivered or, more specifically, not delivered. It is these problems and omissions that I would like to address today. Please consider the following questions carefully before releasing next year's ESPA:
Is this the most opportune time to be testing young children? Today is May 15, which leaves 22.5 days of class time during this school year. Anyone who works with children or even has children will tell you that nine- and ten-year-olds are not primed and ready to take a test at this late date. They have a serious case of spring fever. Expecting them to put forth their best efforts at this time of year is unrealistic.
Have teachers been properly informed and prepared? On April 30, only 15 days ago, I was at a workshop sponsored by the Morris Area Science Alliance. In the course of events, the topic of a speaking portion of the ESPA was brought up. More than three-fourths of a banquet-sized room had not even heard that there was going to be a speaking portion, much less what it would encompass.
A lack of useful information and instruction has been a major problem for weeks now. If it were not for a building principal, my own personal interest, and a need-to-know personality, I would not know half of what I should know at this time. Did the State provide me with all I needed to know about the speaking rubric of the ESPA? The answer is a resounding, no. If I had not taped a program from my home on NJN, I might still be in the dark.
Are there delays surrounding this exercise? I have four days left till I'm expected to administer the ESPA exam. Have I yet received an examiner's manual or any other official useful information to use as an examiner? Again, the answer is, no. When are we going to be fully prepared for an efficient, fair testing of our students?
Who is to answer our questions? We have had many questions over the past few months, and they remain unanswered because every time we ask them we are told, "We don't know yet."
Are unrealistic demands being made? Yes. For example, we have been told that all curriculum-related materials in our classrooms have to be covered or removed. It seems obvious that the architects of this test, no matter how well intentioned, have not spent much time in the professional atmosphere of an elementary classroom.
How many teachers were selected to participate in the preparation and administration practices of this test? To the best of my knowledge, none of the teachers in our district were contacted in the preparation or delivery system for this assessment test. It was certainly not designed with ease or even logic in mind. Parts are downright intimidating. When a workshop participant asked -- regarding security -- what to do if a student regurgitated on a test booklet or answer sheet, she was told that the soiled documents had to be returned to the State.
What is the State doing to address the testing overload in some districts? In our district, fourth-graders are being subjected to five different standardized tests in the spring. They've already taken an Olympiad of the Mind, a Cognitive Abilities Test, the Iowa Test of Basic Skills, a Science Proficiency Test, and now the ESPA. At some point nine-year-olds are going to begin to grow weary of taking tests.
Finally, I close with a series of questions. I am forced to wonder if this test has been created for the children of our State or rather in response to political pressure. I wonder why there was so little preplanning that could have prevented the terrible timing of this test? I wonder why there was so little district and teacher preparation or input?
These and so many other questions need to be addressed by the State in cooperation with teachers, parents, and district coordinators. A measured level of achievement in each grade level is desirable, but I must ask myself and those in a position of influence if this is the most proper climate for the implementation of a valuable statewide test?
Thank you very much for your time.
ASSEMBLYMAN ROCCO: Thank you.
Well, if there is political pressure, it probably came from Rudy. I don't know. (laughter)
ASSEMBLYMAN GARCIA: I think it was internal, John. (laughter)
ASSEMBLYMAN ROCCO: You're saying -- and I had no knowledge of this -- that all of the materials in the room have to be covered or removed?
MR. LAGOON: Yes.
ASSEMBLYMAN ROCCO: And of course we don't do that for the Iowas, the Californias, and the Metropolitans, or whatever.
ASSEMBLYMAN GARCIA: Right. He's shaking his head no, John.
ASSEMBLYMAN ROCCO: Okay. We have Gerry--
UNIDENTIFIED SPEAKER FROM AUDIENCE: It was read to us.
UNIDENTIFIED SPEAKER FROM AUDIENCE: It's in the manual.
ASSEMBLYMAN ROCCO: Gerry, you're saying--
UNIDENTIFIED SPEAKER FROM AUDIENCE: It's written right on there.
ASSEMBLYMAN ROCCO: Well, let's see if we can get some responses. Gerry is indicating that's not the case, today.
DR. DeMAURO: (speaking from audience) It's only-- We only ask that materials not be on display that are specific to what they're testing at the time. So if there is a thing, for example, that-- (laughter) I don't know how to explain it to you. If there is thing, for example, that has to do with the topic at the time, that a teacher can give them all some kind of--
If one kid gets a disadvantage, John, that's where we say that that thing should be handled discretely. We don't have to cover up (indiscernible)--
ASSEMBLYMAN ROCCO: You can't pick it up? (referring to recording microphone) (negative response)
Okay. Would you put that on tape for us, Gerry?
DR. DeMAURO: Like, for example, you don't want to have math formulas--
ASSEMBLYMAN ROCCO: Gerry, you have to-- Could you sit up here with us -- and maybe Jay?
DR. DeMAURO: For example, you don't want to have math formulas--
ASSEMBLYMAN ROCCO: We're trying to get this taped.
DR. DeMAURO: Where do I sit?
ASSEMBLYMAN ROCCO: Gerry, right here. (indicating)
ASSEMBLYMAN GARCIA: Just sit at a microphone.
DR. DeMAURO: Oh, I'm sorry.
ASSEMBLYMAN ROCCO: Would you join us at the table? Maybe at that time, you can respond to some of the questions. (witness complies)
DR. DeMAURO: For example, you don't want to have math formulas on the board and spelling words on the board if that's what's on the test that day.
ASSEMBLYMAN ROCCO: I agree with you.
UNIDENTIFIED SPEAKER FROM AUDIENCE: How do we know?
UNIDENTIFIED SPEAKER FROM AUDIENCE: No, we don't know what's on the test.
DR. DeMAURO: Well, then, you don't have to worry about it.
UNIDENTIFIED SPEAKER FROM AUDIENCE: So we just leave all of our things up?
MR. LAGOON: Have you read your manual?
DR. DeMAURO: Yes.
UNIDENTIFIED SPEAKER FROM AUDIENCE: The manual says to cover it all.
UNIDENTIFIED SPEAKER FROM AUDIENCE: The manual says you have to cover it up.
UNIDENTIFIED SPEAKER FROM AUDIENCE: It's amazing some of the discrepancies that you're-- I'm sorry--
DR. DeMAURO: I think there is a lot of misinformation.
ASSEMBLYMAN ROCCO: Yes, well, the problem is, John, we can't pick it up, and we have to do that because I think there are very pertinent questions, and Jay and Gerry can, maybe, resolve a number of the issues that are out there right now, which are circulating in the community.
J O A N N E M E I N: We've been told that we had to cover everything in our classroom that was instructional material. It was read to us from a manual that we haven't seen yet.
Now, you're answering us that -- only based on what's on the test-- We have not a clue what is on this test. We haven't seen anything. So how will we know what should be covered and what should not be covered?
DR. DeMAURO: Right. On the math day, for example, you shouldn't have formulas for perimeters. You shouldn't have diagrams that might give them an unfair dis -- advantage, rather. That's the kind of thing we're talking about. That's only prudence.
MS. MEIN: Well, it's not what the manual had said. It isn't what was read to us. It said--
DR. DeMAURO: I'm sorry you didn't get your manual. That's as local problem. Your manuals are delivered to your district.
ASSEMBLYMAN ROCCO: Okay. But you're indicating material around the classroom -- only those things that might pertain to a test would be covered, I think which is what most teachers would do anyhow.
DR. DeMAURO: Yes, it's just something most teachers would do anyway.
ASSEMBLYMAN ROCCO: Okay. Well, that's important.
MR. DOOLAN: Assemblyman, if I could ask--
ASSEMBLYMAN ROCCO: Jay.
MR. DOOLAN: Is there a problem? Many of you have not gotten the test materials yet? No one has gotten them?
MS. MEIN: Don't have anything.
UNIDENTIFIED SPEAKER FROM AUDIENCE: No. You do need to know that in the manual -- this is the training copy -- it says that you can receive these up until May the 14th. Their district may have just received them yesterday, and it's only the 15th. In order to disseminate all that material to your teachers, it certainly could not possibly be done in 24 hours, so that maybe a problem from your districts. I know there is a district in Camden County that just received them yesterday.
MS. MEIN: We received--
UNIDENTIFIED SPEAKER FROM AUDIENCE: So their teachers certainly may not have that in their hands.
ASSEMBLYMAN ROCCO: All right. I'm going to have to get control of this. We're not being able to get the recording necessary. So let's continue with those at the mike, and Jay and Gerry will help resolve some of the issues. I'm sure those of you who have questions out there will be able to get the responses from Jay and Gerry.
So who is going to go next?
MS. MEIN: I will.
ASSEMBLYMAN ROCCO: Harriet, is it?
MS. MEIN: No, I'm Joanne.
ASSEMBLYMAN ROCCO: Joanne. All right.
Joanne, do you want to introduce yourself?
MS. MEIN: Sure.
Hi. I'm Joanne Mein. I am also from Rockaway Township in Morris County, and I teach fourth grade.
I'd like to thank you for the opportunity that you've given us today. You've taken time out from your busy schedule, and we, as fourth-grade teachers, really do appreciate that you've done this for us.
In addition to being a teacher in Rockaway Township for five years, I'm also the parent of three children. My older children have taken the Early Warning Test three times in preparation for the High School Proficiency Test. My youngest child, a sixth-grader, has already had class time dedicated to preparation for the EWT.
As a parent and as a teacher, I have lived through the anxieties of my children and my students as they have faced the prospect of these tests. I can tell you candidly that as the State of New Jersey increases the number of tests our students must endure, their anxieties also increase.
As a parent, I am deeply concerned at the lack of information that parents have received about the new fourth-grade test. I am equally concerned that when asked legitimate questions by parents, I, as the teacher, essentially have no answers for them.
Why, you wonder? If the Department of Education has extensive information about the contents of the test, about the proper procedure for testing, about the grading, and about the follow-up, why hasn't that been shared with me? Why doesn't my school district have the answers to questions that parents and I will ask?
I am equally disturbed that my kindergarten through third-grade teaching colleagues have no information about this test. I'm sure you realize that we will not simply be testing the fourth-grade curriculum, but, indeed, will be testing all that came before it. Every teacher should have information about the State's testing program, and they do not.
As an educator, I must question the amount of time this test will take my students to complete. As has already been stated, this test will take 11 hours, this year. How many hours will this test require when the State adds the social studies, foreign language, arts, health, and phys ed components? Does it make sense that the youngest children being tested by the State, the children with the shortest attention spans, will be taking the longest test? The logic escapes me.
Another area of grave concern on my part and on the part of my colleagues is the lack of proper teacher preparation and training to administer this test. I have an advantage over my colleagues. I was selected to attend a training session on the public speaking component of that test. My colleagues were told to view a two-and-a-half hour video and then attend a two-hour session where I and others would share the information we received at our training. As a teacher, I must tell you that is a terrible way to train those responsible for administering this test. How will the Department, in all good conscience, compare the scores of students across the State, when the majority of teachers have not had the same opportunities for training?
You have heard a great deal about the public speaking component. I would like to give you a picture of the mathematics section of the test. As you may or may not know, manipulatives are part of the test. I sit here today, less than a week before this test is to be given, and I have not seen these manipulatives.
Why does that concern me? Because I've been told that I must punch them out of the cardboard sheets, place the pieces in Baggies, and then distribute them to my students. When I am expected to do this task? When will I get the manipulatives? Why aren't I being provided with all my materials in a state of readiness?
In addition, I have been told that my students are not allowed to spend any time getting acquainted with their manipulatives. Every classroom teacher, every expert in this field, will tell you that children should -- no, must -- be given time to be comfortable with manipulatives or they will surely struggle at the task before them.
This, and many other concerns, prompts me to wonder who created this test. I hear rumors that teachers were involved. Who are they? Are they currently teaching fourth grade? Have they even taught fourth grade? I would welcome the chance to meet these people and hold a very candid discussion with them. I would ask them to come to my school and administer this test from beginning to end. Once they have done that, I think they will agree that serious errors have been made.
Finally, I am curious about the procedures, if any, that are in place for teacher feedback on this pilot. Will I and my colleagues from across the State have an opportunity to articulate our concerns about the test? Will we have the chance to discuss the pros and cons? Will anyone be scheduling such opportunities? I hope so, because, ladies and gentlemen, if that does not happen, I can assure you that this fourth-grade test will have been a terrible waste of time and money for everyone concerned.
ASSEMBLYMAN ROCCO: Thank you, Joanne.
Jay, do they have a system of getting feedback?
MR. DOOLAN: Yes.
ASSEMBLYMAN ROCCO: Do you want to tell us?
MR. DOOLAN: There is a system that's included with the materials, and we'd like to have the feedback. Based on the information we're getting this afternoon, we've heard in calls throughout the State, we'll make a concerted effort to get more focused feedback not only through the actual written, but through other means, as well.
MS. MEIN: So there might be a meeting such as this afterwards, where fourth-grade teachers could come and discuss--
MR. LAGOON: Or can the State Department--
DR. DeMAURO: These meetings go on all the time. We have eight, right now, standing Committees. Most of them are fourth-grade teachers who have been solicited from around the State. We have hundreds and hundreds of resumes that we've called on. Each Committee has on it-- For example, the Content Committees each seat about -- there is a 25 or 30--
UNIDENTIFIED SPEAKER FROM AUDIENCE: Eighteen.
DR. DeMAURO: There are 18 people, somewhere in that range. And it does for each one of the Content Committees-- The Sensitivity Committees, that's where parents are seated, as well as fourth-grade teachers. And we have, as well, Technical Advisory Committees, psychometric experts from around the country, the best in the field.
ASSEMBLYMAN ROCCO: You're talking about Content Standards Committee or the testing committees?
MR. DOOLAN: No, testing committees.
DR. DeMAURO: No, separate committees -- fourth-grade teachers--
ASSEMBLYMAN ROCCO: All right.
DR. DeMAURO: There is a District Advisory Group of supervisors and principals that meets regularly and advises on this test. There is an Advisory Committee that started out as developmental psychologists and cognitive scientists, who designed the test, and now has included in it district personnel. There is a Steering Committee that evolved from that Advisory Committee that advises on the testing parameters for children of this age. There is a Special Education Committee of special educators -- fourth-grade teachers -- who have been meeting for about two years now on the design of this test. This is public-- If you really want to know who is on the test -- which fourth-grade teachers -- I can give you hundreds of those names. It's public. It's a matter of public--
ASSEMBLYMAN ROCCO: Rudy.
ASSEMBLYMAN GARCIA: May I ask a question? Was there also a Committee of teachers, bilingual educators?
DR. DeMAURO: Yes.
ASSEMBLYMAN GARCIA: And the limited-English proficiency students--
MR. DOOLAN: Yes, absolutely.
DR. DeMAURO: Not only are they infused into the Content Committees, but as well, they sit on the Sensitivity Committee where they review materials for cultural offensiveness, stereotyping, and the empirical analysis we do for item bias.
ASSEMBLYMAN GARCIA: How many teachers, bilingual educators were on this Committee?
DR. DeMAURO: There is about two or three on each of the Content Committees and about four or five for Sensitivity. Also, we've met before the Bilingual Advisory Committee, I think, three times now on the test -- who set up an Assessment Committee to advise us, specifically on bilingual education.
ASSEMBLYMAN GARCIA: And limited-English-proficient students are going to have to take the speaking portion of this test, as well?
DR. DeMAURO: Limited-English-proficient students-- There is a policy of voluntary district exception that has been in place since October 6, 1993 -- from the State -- and that's still in force. Those who want to take the speaking portion of the test in their native language may do so.
ASSEMBLYMAN ROCCO: Okay. Thank you, Joanne.
I think we'll go to Superintendent John Polomano and come back to Pat White and Harriet--
UNIDENTIFIED SPEAKER FROM AUDIENCE: We're not (indiscernible) they representing us.
ASSEMBLYMAN ROCCO: They were representing you all?
UNIDENTIFIED SPEAKER FROM AUDIENCE: Yes.
ASSEMBLYMAN ROCCO: Okay. We can cross you off.
UNIDENTIFIED SPEAKER FROM AUDIENCE: Thank you.
ASSEMBLYMAN ROCCO: Thank you for coming.
John Polomano, Audubon Schools.
ASSEMBLYMAN GARCIA: Mr. Chairman, just one more question.
ASSEMBLYMAN ROCCO: Rudy has another question.
ASSEMBLYMAN GARCIA: Does it take 11 hours to administer this test?
DR. DeMAURO: That's a matter of district--
MR. DOOLAN: Gerry, speak in the mike.
DR. DeMAURO: I'm sorry. That's a district preference. How they administer the speaking test could take more or less time depending upon what's most convenient to them. Two mornings take about two and a half hours, and two mornings take about an hour and a half. And then, the speaking part is up to them. They could do it all in a day or they can do it over several days. It's how they want to do it, at their own convenience.
ASSEMBLYMAN GARCIA: And the other question: Are either of you two going to sit in on one of these tests to get a firsthand look at this?
DR. DeMAURO: Yes, the staff-- We've taken both of our staffs and sent them all out for every day of testing. We have to have somebody in the office in case of a security breach and things of that sort. So the whole staff can't be out there.
ASSEMBLYMAN GARCIA: Then neither one of you have yourselves?
DR. DeMAURO: I've been in the classrooms, yes.
ASSEMBLYMAN GARCIA: With this test already?
DR. DeMAURO: The test was built on a series of observations to start with, and we went out in classrooms then.
ASSEMBLYMAN GARCIA: Okay.
MR. DOOLAN: I should just add, Assemblyman, that the reason for the length of the test this year is the fact that it is a field test. The actual test will not be as long as it is for this year for the field-testing purposes, because we're adding items -- much more items than would be in the actual test, in order to identify those items that are problematic, those items that are inappropriate, and that sort of thing. So it is longer by design, as most of you know, in terms of a field test situation.
UNIDENTIFIED SPEAKER FROM AUDIENCE: Therefore, it's invalid. It would be invalid. It's too long, that's what we're saying.
ASSEMBLYMAN ROCCO: Let's go with John Polomano, and then we can get--
If you would like to speak at the end when we're finished with the people who have signed up--
J O H N P O L O M A N O JR.: Good afternoon.
John, thanks for coming here this afternoon.
ASSEMBLYMAN ROCCO: Just give your name for the record.
MR. POLOMANO: John Polomano, Superintendent of Audubon Schools.
It's good to see you again.
Assemblyman Garcia, I've heard a great deal about you, and it's good to see you in our neck of the woods.
Members of the Department, thank you very much for your time.
I think some of the confusion-- I have a statement that I sent to Chairman Rocco, and he can distribute that to you at a later time, and I will read that into the record.
ASSEMBLYMAN ROCCO: By the way, John sent, as he has indicated, very good information and very good issues clearly identified, with some recommendations.
You may want to send that to Jay and Gerry, as well.
MR. POLOMANO: I will.
Jay, I think some of the things that we are hearing here -- and to the members of the Committee -- you're probably as frustrated this afternoon as we have been for five or six weeks. It seems that we hear new information each and every day.
For instance, I'm astounded that there was not an awareness that there would be extra costs to the districts because two certified individuals would have to observe these speeches. I'm really astounded that no one had an awareness of that. I think it's clear that the speeches have to be observed by two people. You cannot use an aide -- who may not be a certified person -- or a volunteer. It's specifically in there that it has to be two certified people, and no one was aware that there was an additional cost. That is astounding to me.
We're a very small district, as John knows. Carmine knows we're a K-to-12 districts, and we have two elementary schools. We don't have folks who are unassigned, like many of these districts don't, especially at this time of year.
ASSEMBLYMAN ROCCO: Could we just deal with that issue, just for a second, John?
How do you expect districts to deal with the certification of the observers?
DR. DeMAURO: I'm not sure I understand.
ASSEMBLYMAN ROCCO: In other words, there has to be two certified personnel observing.
DR. DeMAURO: Yes.
ASSEMBLYMAN ROCCO: How do you expect the districts to deal with that, when you were putting this together?
DR. DeMAURO: There are several models they can choose from. We wanted to -- because of exactly the issues that you mentioned before about reliability--
ASSEMBLYMAN GARCIA: With Charley.
ASSEMBLYMAN ROCCO: Such as?
ASSEMBLYMAN GARCIA: What are the models?
ASSEMBLYMAN ROCCO: E.g., i.e.?
DR. DeMAURO: Oh, basic skills teachers, speech pathologists, library media specialists.
MR. POLOMANO: May I ask what happens to their responsibilities when they're assigned?
ASSEMBLYMAN ROCCO: They have to be taken from another job--
DR. DeMAURO: Yes, that's right.
ASSEMBLYMAN ROCCO: --at the time to fill in to be the second observer.
DR. DeMAURO: That's right, as is true of any test administration using teachers.
MR. POLOMANO: It's not correct.
ASSEMBLYMAN ROCCO: Well, I don't think any other test requires an additional person to be in the classroom.
DR. DeMAURO: Often, for example, the State tests in eighth and eleventh grade-- For every 25 students there has to be a proctor present.
MR. POLOMANO: A proctor.
ASSEMBLYMAN ROCCO: Well, they have a proctor.
DR. DeMAURO: Certified person.
MR. POLOMANO: Yes, a teacher.
ASSEMBLYMAN ROCCO: Yes, well, they have a teacher. So how do we get--
DR. DeMAURO: With the test administrator.
ASSEMBLYMAN ROCCO: Oh, you're saying in addition to?
MR. DOOLAN: Yes.
DR. DeMAURO: Yes. This is standard practice.
MR. POLOMANO: Yes, that test administrator does not have to be in that classroom. If you are an eighth-grade teacher and we can, as someone said, have our eighth-grade students in one location or if we keep them in their homerooms, their homeroom teachers, or whomever, will proctor that test. You're not asking for another individual to come in during the entire length of that test and in this area -- the speech portion of the test -- to help with the assessment.
We're not arguing the validity of that; although, I don't like the concept of public speaking. But I'm asking how there could not have been the understanding that somewhere--
DR. DeMAURO: That's was understood.
MR. POLOMANO: Excuse me. That somewhere along the line, someone will have to be removed from their responsibilities in order to help proctor these tests -- in order to score, not only proctor-- Proctoring is one thing, as you know, scoring is another. You're asking for two independent scores.
And while we're at it, am I understanding you to say that, in effect, this test -- the speech part, the public speaking part of the test you're not looking at, at this point, but yet, we have to secure the materials. You're telling us that we have to secure the materials, we have to score each child, and we have to return those materials to you.
DR. DeMAURO: No. Not speaking, you keep it.
UNIDENTIFIED SPEAKER FROM AUDIENCE: Oh, it says right there--
DR. DeMAURO: Oh, we're collecting it? (affirmative response) I'm sorry. I was wrong. We're collecting the prompt this year. You're right.
MR. POLOMANO: You're collecting the prompts. Are you collecting the score sheets, too?
DR. DeMAURO: Yes, and the score sheets, too.
MR. POLOMANO: Now, again, I understand that you're extremely busy. I don't understand how you could not know that. And here we're being told-- And I'm a district superintendent, I tend to think like all the educators here -- that we really make an effort to be informed on what's going on -- and I've spoken to you. You've been very cordial when I've spoken to you, and I really appreciate that, but here is the frustration that we've had. That we've gotten different answers, we've told our staffs many different ideas, and now there seems to be a discrepancy. We understand, as you do, that we have to return the score sheets.
DR. DeMAURO: And she's right.
MR. POLOMANO: Are they being used or not being used?
DR. DeMAURO: The score sheets have to be returned because we have to study the administration of the test. Now, there are also some experiments going on around the State. We're going to study that in a different way. But we have to do correlations. We have to see the type of teacher and how that affected scores, whether there is a variance in scoring that is attributable to certain factors. We have to know that.
MR. POLOMANO: But you're also saying they were seeing types of scoring, types of teaching -- never-- I think I've tried to follow the development of this test pretty well, and never was I under the understanding that there was a public speaking portion of this test, to be implemented in a field test situation this year, that was going to be sent back for you to experiment or for you to analyze.
DR. DeMAURO: Analyze, yes.
MR. POLOMANO: I understand the core curriculum standards and that we are expected to teach those standards. I don't have a real disagreement with that. Now, I'm telling--
ASSEMBLYMAN ROCCO: Go ahead, John.
I guess I still haven't felt comfortable with the issue of the additional personnel. In an elementary school, where you have very few additional people around to begin with, those people have to come out of the library, they'll have to come out of other parts of the building in order to meet the responsibility -- or substitutes have to be brought in -- so there is a cost factor here that districts have to bear. It does become a problem. I think that's an issue that you really have to look at very, very carefully. Even though in eighth grade, you're indicating -- and a proctor has to be available--
In the elementary schools, it's been one teacher giving the test with her students, self-contained environment, not a departmentalized situation. It's totally different than the secondary program. So, in that case, it seems to me that we are adding additional cost to these districts, and I think that's something the Department has to look at carefully when they're talking about two certified people coming in.
Take that back to the Commissioner, Mariann, and I'll talk to the Commissioner about that, as well.
There has got to be a better way to do that.
MR. POLOMANO: Also, some of the other -- I guess I would call them, politely -- missteps that have taken place-- And I would tend to think I would probably be really called on the carpet as a district superintendent if I took some of these missteps.
First, yes, that program was televised twice. I think because there was lack of foresight that many schools were closed on April 5 and 7 when it was first televised because of spring recess. It was not until it was called to the attention of the Department that this was an inconvenient time that it was retelecast.
Also, something I was really quite concerned about was -- and I would ask anyone, if they will-- I haven't been able to find, when I watch the tape, a disclaimer that the following children or child actors that are acting out on one, two, three, four-- Maybe I didn't pick that up.
DR. DeMAURO: It's in there.
MR. POLOMANO: Is that said by the commentator?
DR. DeMAURO: Yes, it's in the tape.
MR. POLOMANO: Is that said by the commentator, that these are children actors?
DR. DeMAURO: It's said by Roseann Harris of our staff. (phonetic spelling)
MR. POLOMANO: Because I think many of us got the impression that these were classroom children.
DR. DeMAURO: No. We didn't want to hold any child in the State-- See, we had to have children to exemplify all the score points -- zero, one, two, three, four. In order to do that, that means that we would have to hold the speech of some child up as an example of poor speech. We didn't do that. What we did was we observed in the classrooms. We piloted in the classrooms. We scripted what the kids said and then hired actors so within the same student they would have an example of the good score point and a low score point, because we didn't want to hold any child in the State as an example of poor speech.
MR. POLOMANO: I don't think there was ever an explanation of how that process was done on the tape.
UNIDENTIFIED SPEAKERS FROM AUDIENCE: (noise from audience)
DR. DeMAURO: It's not a secret, first of all. But in the tape she said they were actors.
MR. POLOMANO: The point is I think a number of people got the impression that this could be--
ASSEMBLYMAN ROCCO: Let's just address the Chair so that we're not talking back and forth. It will be easier for you, Gerry, too.
MR. POLOMANO: The impression, John, that we got in the tape was here is how it can be done in the classroom, at least that's the impression I received. I think that made me-- I was surprised that they were actors. Here it is, here's how it can be done, here's what you can do, and here's what these children have done after an hour. Also, interestingly enough, one of the teachers mentioned the lack of information.
I know what the Eighth-Grade Warning Test and the Eleventh-Grade HSPT, the MBS before that, the Ninth-Grade HSPT-- We've been through all the initials. I know we always received a form letter to send to parents explaining the importance of this test. I don't know if the Committee is aware that we just got that letter -- an example letter -- to send. We had it faxed to us. Laurie Garcia (phonetic spelling) and the county office have been tremendous working with all of the concerns here. We just received a fax of a letter that we could send to parents and were told that the packets would arrive next week. The tests are to begin on the 19th.
ASSEMBLYMAN ROCCO: John, do you think -- we have other testimony here, as well-- Is your feeling that the -- what you're expressing is true throughout this county?
MR. POLOMANO: I would believe so.
I would also indicate that we have long been an advocate of eliminating standardized testing in our district. I think I see the State finally catching up to us and saying now we're going to have fourth, eighth, and eleventh. I applaud the efforts. I applaud the core curriculum standards and an assessment type of situation.
The timeliness of this issue is deplorable. I cannot believe that teachers saw a final copy of this test and said, number one, yes, and number two, administer it in late May and early June. I don't think that that was ever said. I know that a teacher did say, if you were going to have a standardized test, don't give your CAT test or any other standardized test. Replace this and have it in May because it will reflect. No teacher said give it a field test after students have been tested over and over already this year.
Now a couple of other points I wanted to bring out-- Someone mentioned the manipulatives. Am I to understand that the ruler is incorrect on that manipulative? Is the zero missing or something on the ruler of the manipulatives?
UNIDENTIFIED SPEAKER FROM AUDIENCE: It's not missing. It's the way it was designed to be.
DR. DeMAURO: That's the way it's designed to be.
MR. POLOMANO: That's the way it's designed to be.
UNIDENTIFIED SPEAKER FROM AUDIENCE: It's designed to measure what a student knows (indiscernible).
MR. POLOMANO: Okay, and even though it's designed that way, there is no concern that the student should in fact -- and research shows this -- become familiar with those manipulatives before the test?
DR. DeMAURO: The materials you're talking about are secure.
MR. POLOMANO: I know.
DR. DeMAURO: It's very hard for me to discuss these publicly right now. But when you see them--
MR. POLOMANO: I respect that.
DR. DeMAURO: --you'll see that your concerns are unfounded.
ASSEMBLYMAN ROCCO: John, he could tell you, but he'd have to shoot you. (laughter)
MR. POLOMANO: They'll go along with the bulletin board police who are going to monitor what's in your classrooms on the days of the test.
DR. DeMAURO: The tape will self-destructive in five seconds. (laughter)
MR. POLOMANO: John, I wanted to mention also-- You know I've been really-- I've mentioned to you a number of times about expense, and that is a big concern.
Gerry, has anyone assessed what the total cost of this test is? By that I mean, the cost of training, the cost of substitutes, and the total dollar value placed on lost instructional time by individuals who will now be expected to go into the fourth-grade classrooms to cover?
I also believe that a copy of this test should have been give to Child Study Teams across this State before ever being implemented, to have them see it in order to develop IEPs before the fact not after the fact. I really believe strongly in the core standards. I believe an assessment is really necessary.
I believe -- and I have a list of recommendations that I did forward to Assemblyman Rocco -- that we should delay the field-testing to September 1997; permit adjustment for the IEPs; eliminate the public speaking portion of this test at this grade level; develop statewide training to score the public speaking if you go with it, similar to the SA training that's done for people who are going to score the SREs; and implement an initial testing in the spring of 1999.
Again, I applaud your efforts in trying to secure information that is going to be important to make a decision, but I believe it's important to make a decision for this test administration, the practice test. If, in fact, we want to have an instrument that we can use for an analysis -- and I applaud that -- follow what we did with the Eighth-Grade Warning Test and Eleventh-Grade HSPT -- a number of years of testing before we finally put it into place as the real test. That was good. We had the opportunity to align our curriculum that way, not teach to the test, but align the skills that students could transfer from year to year.
Again, I thank you, and I will await a response to a letter that I faxed saying that Audubon has chosen not to participate in this year's pilot test.
ASSEMBLYMAN ROCCO: Thank you, John.
ASSEMBLYMAN GARCIA: Mr. Chairman, may I just ask a question?
ASSEMBLYMAN ROCCO: Rudy, yes.
ASSEMBLYMAN GARCIA: Did you cost out how much it would cost for the training, all the additional personnel statewide?
DR. DeMAURO: For the State components of it, but the districts have a lot of leeway in how they can administer the speaking part of the test. That would be hard for us to cost out.
ASSEMBLYMAN GARCIA: For the State component then. How much was it?
DR. DeMAURO: We had-- The original bid from the contract was something like $700,000.
UNIDENTIFIED SPEAKER FROM AUDIENCE: It's 200-some (indiscernible) for this year, but it's less than that--
DR. DeMAURO: Yes, it's a lot less than that.
ASSEMBLYMAN GARCIA: So what is it?
DR. DeMAURO: It's under-- We don't have all the bills yet, but it's somewhere in the--
UNIDENTIFIED SPEAKER FROM AUDIENCE: It was costed originally at $206,000. It will be less than that because we did a lot of the work prior to the contract (indiscernible).
DR. DeMAURO: It's $206,000 because that included -- because we did a lot of the work ourselves.
ASSEMBLYMAN GARCIA: And that's the cost to the Department of Education?
DR. DeMAURO: Yes.
ASSEMBLYMAN GARCIA: And how about the cost to the local school districts?
DR. DeMAURO: Well, that would be difficult to get a handle on because they implement under different models -- using different kinds of teachers.
MR. POLOMANO: Are you asking for that total? At the end of the test are you going to ask "What was the cost to you," and also factor in lost instructional time? If I have a teacher who has lost classroom instruction, in effect, I'm paying that teacher that teacher's per diem rate to sit in the classroom and be a second assessor. I've lost instructional time in other classes. We all know no matter how effective a substitute teacher is, it doesn't take the place of regular classroom instruction.
Also, I think-- I don't know if you're aware, too, some of the suggestions that were given -- and you can give these speeches maybe five at a time -- send the students somewhere to do their training-- In other words, to get their prompt and to do their training, I don't know where we're going to send them without direct supervision. An option for a small school district like that is not to send them to a library under a certified librarian because we don't have them. So send them somewhere to do the training for an hour, come back and give their speeches-- Again, some of the concepts had some validity, but some of them were really not useable.
I think, though, the direction has to be-- In addition to the future of the Fourth-Grade Test, what are we going to do this year? There are still a number of concerns for this year yet.
ASSEMBLYMAN ROCCO: Am I understanding? When they go to the library or wherever they're doing the research, are they allowed to research that item while they're there?
UNIDENTIFIED SPEAKERS FROM AUDIENCE: No.
ASSEMBLYMAN ROCCO: They're not permitted to research the item that they're going to talk on? Is that correct?
UNIDENTIFIED SPEAKERS FROM AUDIENCE: No.
DR. DeMAURO: They prepare visuals usually.
ASSEMBLYMAN ROCCO: Pardon me?
MR. POLOMANO: There is a suggested list of materials, Assemblyman Rocco, that we're told that the students can use. I think index cards, if I'm not mistaken, construction paper-- For instance, they can't go to my library, go on the Internet, get some items off of there, and prepare their speech.
DR. DeMAURO: Right.
MR. POLOMANO: However, those students, if they're sent to the library, are supposed to be supervised. Right?
DR. DeMAURO: Yes.
MR. POLOMANO: How does a small district--
ASSEMBLYMAN ROCCO: And what is it they're doing in the library for the hour?
DR. DeMAURO: When we went out and field-tested it, mostly the children colored. They had things-- They drew pictures of what they were going to speak on. It's a two-minute speech, and they usually make visuals.
ASSEMBLYMAN ROCCO: So they're doing--
DR. DeMAURO: They make visuals. Boxes. Kids illustrated with boxes.
ASSEMBLYMAN ROCCO: Things that they were going to use to -- props, so to speak. Make props, etc.--
UNIDENTIFIED SPEAKER FROM AUDIENCE: (indiscernible) in the room. Some small districts put two of their fourth-grade classes together. You have two teachers. You don't need anybody else.
MR. POLOMANO: You're correct. There are options that were available. That's correct. But, again, when we're talking about the preparation -- that's an option we're using, preparing in a classroom, because, again, some of these options have to be realistic, too. You do not send-- Most folks here who are in classrooms and who have taught know that you cannot send five students unsupervised somewhere. And it's not an option for many districts to send them to a library. We don't have certified librarians, and the code is quite clear that those students have to be supervised by a certified individual. So some of the options are nonexistent.
UNIDENTIFIED SPEAKER FROM AUDIENCE: And some of the options are that they can prep in the classroom. You can do all of your students in one day.
MR. POLOMANO: Absolutely. And one thing that remains clear--
UNIDENTIFIED SPEAKER FROM AUDIENCE: So there is-- I'm just trying to point out there are many options. You can choose what--
MR. POLOMANO: You're correct. One thing that remains clear is those speeches have to be always observed by two certified people.
UNIDENTIFIED SPEAKER FROM AUDIENCE: Yes, yes. And in some districts, they're putting two fourth-grade classrooms together to allow two fourth-grade teachers in that building--
ASSEMBLYMAN ROCCO: Okay, John, thank you.
MR. POLOMANO: Thank you.
ASSEMBLYMAN ROCCO: Audubon has decided not to participate. Is that correct?
MR. POLOMANO: We're waiting for an answer on that. We did fax a letter and said that we thank you for the opportunity and do not wish to participate in this year's pilot.
ASSEMBLYMAN ROCCO: Oh.
ASSEMBLYMAN GARCIA: Mr. Chairman.
ASSEMBLYMAN ROCCO: That was very polite, John. (laughter)
MR. POLOMANO: Uncharacteristic, John. (laughter)
ASSEMBLYMAN GARCIA: Just one question.
ASSEMBLYMAN ROCCO: Rudy.
ASSEMBLYMAN GARCIA: After the school districts have finished administering the test, whatever the costs are, has the Department-- Do you foresee reimbursing the school districts for those costs? (laughter)
MR. DOOLAN: No.
DR. DeMAURO: No.
ASSEMBLYMAN GARCIA: So the Department is not going to repay the districts for any of the costs associated with administering this test. How are they going to fund it then? They have to absorb that themselves?
DR. DeMAURO: Those districts, in the long run, will save money on the use of commercial instruments. No matter what year we field test, if the commercial instruments are being given to some person -- for example, monitoring or Title I -- that's the year you'll do double testing, unless you want scores based on a field test. And that's the very thing the districts don't want.
ASSEMBLYMAN GARCIA: I'm just saying for this year. Let's talk--
DR. DeMAURO: Well, this year is the field-test year.
ASSEMBLYMAN GARCIA: Right. And this field-test year where there is additional costs, are the districts going to be reimbursed by the Department for the expenditures associated with administering this test?
DR. DeMAURO: Not to my knowledge.
MR. DOOLAN: No.
ASSEMBLYMAN ROCCO: I think that's pretty clear, Rudy. (laughter)
Barbara Michalsky, Assistant Superintendent, Pitman Public Schools, another one of our very cost-efficient districts.
B A R B A R A V. M I C H A L S K Y, Ed.D.: Thank you. Good afternoon, and thank you again for giving me this opportunity to speak to you about the ESPA.
My name is Barbara Michalsky. I'm Assistant Superintendent of the Pitman School District in Pitman, New Jersey. Today, I am here to provide testimony regarding the Elementary School Proficiency Assessment, known hereafter in this document as ESPA.
NJASA and most of the New Jersey school districts wholeheartedly supported the State Department of Education efforts to establish pupil performance standards that are challenging, rigorous, and internationally competitive. Many districts have initiated a variety of labor-intensive curriculums, instruction, and staff development activities aligned with the core curriculum content standards.
However, NJASA has repeatedly pleaded with the State Department of Education to give districts sufficient time to effectuate these very crucial and time-consuming changes. Our appeals were not heard.
I would like to discuss six areas of the ESPA that have caused concerns at the district level:
First, staff development concerns and issues had to be addressed by each district. Districts were asked to take fourth-grade teachers to training in mid-April for a mid-May test date. Teachers were given a half-day workshop on the speaking portion of the test; given a rubric to use, which they had never seen; and asked to view a video, which was vague. Often, the raters on the video did not agree to the score given to each child.
Also, district personnel were asked to attend a half-day workshop where security measures, the length of the test, and the directions for administration were discussed. Again, this was lengthy and time consuming. The presentation was given in a room that was overcrowded and information packets were not available for each participant. Total time expended out of district was two administrators for half a day, one administrator, and six fourth-grade teachers for half a day. The total cost to my district was $2655.
Furthermore, staff development had to occur at the district level. I gave a half-day workshop for all fourth-grade teachers, BSI teachers, SE staff, and other selected personnel -- 15 total -- to view the video, discuss the rubric, and discuss the actual test administration. I provided substitutes for the personnel -- total cost, $1950.
Second, the State Department did not provide enough packets of information for the personnel being trained. Additional material had to be duplicated -- total cost, staff time and paper costs, $245.85.
Third, the ESPA itself represents a test that is unfamiliar to staff. No one would disagree to give a test with reading, writing, and mathematic sections that closely align with EWT or HSPT for the first administration. However, the State has added science and a speech portion. The science portion represents an area of concern to all elementary teachers. If time had been given, content could have been aligned to the standards and additional staff development could have been scheduled for each district in the science area.
The speech portion of the ESPA represents an area new to all staff. Teachers require students to speak in classrooms, but to ask them to use a rubric that is new and to train teachers to use the rubric in less than one month is an administrative nightmare. Again, the idea is good; however, the timing is poor. The total cost, staff training -- three hours times eight staff -- $1040. A central office administrator spent four days coordinating, planning, and implementing the staff development activities -- total cost, $1450.
Fourth, the length of the ESPA is not in line with a fourth-grader's ability to concentrate. To ask a child to sit for that length of time in May is difficult. The length of time surpasses that of the SATs and the GREs. Most of the staff must test in the morning and several afternoons to fit it into busy end-of-the-year schedules.
Fifth, the time of the year that the test is given also presents a problem for most districts. The Pitman School District finished a week of standardized testing less than a month ago. All field trips, planned in early/late October 1996, are scheduled for those three weeks: trips for the safety patrol, Gifted and Talented Programs, and regular grade level field trips. It is difficult to ask children to concentrate in late May; the weather is warm and there are a myriad of activities going on in each of the elementary buildings.
Sixth, the State sent a parent information letter -- one copy -- on Wednesday, May 14, 1997 with this information -- and I quote -- "Additional copies will be sent next week," the week the test is administered. If parents are to know about the test, the information must go home by Friday, May 16, prior to the first date of the testing on Monday, May 19. Cost for me to reproduce this packet of information was $76.50.
Again, I would like to reiterate, I support the State Department's efforts to establish a challenging, rigorous test for all fourth-graders; however, I question the introduction of a science section without proper planning; the introduction of a speaking portion with no time for training; the time of the year for the test; the length of the test; the additional staff development needed for fourth grade and other selected personnel; and most important, the financial obligation by a small district that is already fighting for every penny. A total cost to my district, for 118 fourth-grade students, was $7347.35. This total is very disturbing, but again, we will try to adhere to the State Department of Education time line for the new ESPA.
Thank you very much.
ASSEMBLYMAN ROCCO: Thank you, Barbara.
ASSEMBLYMAN GARCIA: Barbara, I'd like to thank you for preparing this information for me, because this information will be very useful to determine and calculate how much the Department of Ed is costing the local school districts. So thank you very much.
DR. MICHALSKY: Thank you very much.
Again, I would like to say we are a very, very small district. So I am assuming that the cost would be several times greater for a large district.
ASSEMBLYMAN GARCIA: Well, in the city where I live we have about 1200, almost 1300 fourth-graders.
DR. MICHALSKY: Thank you.
ASSEMBLYMAN DeSOPO: Barbara, I also want to commend you. This is an excellent job. You seem to have captured the sentiments of a lot of people here, and it's great that you were able to spell it out. Were you a fourth-grade teacher, too?
DR. MICHALSKY: I was a fourth-grade teacher, but I guess it's near and dear to my heart, because I've spent a lot of time over the last month working with my staff so that they will not be frustrated in a test that I think has not been given the proper training in staff development.
ASSEMBLYMAN DeSOPO: I just want to say, Chairman Rocco, that this particular document has probably a great deal of validity and reliability. (laughter)
ASSEMBLYMAN ROCCO: Yes.
ASSEMBLYMAN DeSOPO: And I think it also could be measured. (laughter)
ASSEMBLYMAN ROCCO: Thank you.
DR. MICHALSKY: Thank you.
ASSEMBLYMAN ROCCO: Thank you, Barbara.
We have Dr. Odete Silva, Assistant Superintendent, Irvington.
Did you come down from Irvington North?
O D E T E B. S I L V A, Ph.D.: Yes, I did.
ASSEMBLYMAN ROCCO: Wonderful. Thank you for coming. That's a long trip, as well.
DR. SILVA: Good afternoon, Chairman Rocco, and members of the Education Committee of the Assembly.
My name is Odete B. Silva. I'm Assistant Superintendent of Schools in Irvington, which is an urban district in Essex County. I'm also the Chairperson of the Curriculum and Instruction Committee of NJASA.
NJASA has wholeheartedly supported the State Department efforts to establish pupil performance standards which are highly challenging, rigorous, and internationally competitive. To this end, many school districts throughout the State of New Jersey have initiated a wide variety of labor-intense curriculum, instruction, and staff development activities aligned with the content standards.
However, NJASA has repeated pleaded with the Department to give us sufficient time -- two or three years -- to effectuate these very crucial and time-consuming changes before the State assessment schedule kicks in. Our appeals were not heard.
The State Department of Education proceeded with the formulation of an assessment schedule which will call for a fourth-grade test in the targeted areas of reading, writing, speaking, mathematics, science, and cross-content, to be administered beginning with the week of May 19, 1997. This unreasonable, illogical, and educationally unsound timetable for this test has been compounded by the fact that as of this late date, the State Department of Education has yet to disseminate to the districts any information regarding the content of each tested area, the respective areas test item specifications, the performance criteria, test samples, scoring rubrics, and frameworks.
Additionally, the State has not held meaningful training sessions for grade-four teachers on the administration of the open-ended scoring rubrics and speaking components of the test. A televised presentation and a videotape sent to the districts constituted the primary State Department's interpretation of meaningful training. And as you have heard from many colleagues from across the State, it takes much more than that to prepare staff and students.
To add to the confusion, unconfirmed statements are being made throughout the State amongst the education circles that the science field test may not be administered this year.
In view of all these serious problems, we requested that the State Department postpone the plans for field test of ESPA-four to 1997-1998, and that the State core curriculum content standards State assessment schedule be adjusted accordingly.
Our request falls within the most basic principles of education research in the areas of curriculum, instruction, professional development, and student performance. Our request was not granted, which means that State Department officials do not quite understand the relationship between assessment, curriculum, and professional development. In other words, it seems that the State Department took the assessment as the primary focus of the entire process.
As a matter of fact, back in February of 1996, we testified -- the Association testified -- before the State Board of Education at the time the Commissioner was presenting his platform on the content standards measuring out to the 21st century. We requested -- knowing that the State was going to approve a set of rigorous standards and, at the same time, rush into an assessment process-- We requested, at that time, that between the approval of the standards and the first administration of any of the assessments that at least two or three years be allowed for all districts to adjust curriculum, to provide for instructional training for allowing teachers to get the proper training -- the time that's required to close the bridge between curriculum, instruction, and assessment.
We do not want our kids to be the guinea pigs of the State Department or for other reasons to be tested when we feel that we need more time to close the gap between curriculum, instruction, staff development, and then, the assessment piece.
ASSEMBLYMAN ROCCO: Thank you, Dr. Silva, and thank you for coming all the way down.
ASSEMBLYMAN DeSOPO: Thank you.
ASSEMBLYMAN ROCCO: We very much appreciate that.
DR. SILVA: Thank you very much.
ASSEMBLYMAN ROCCO: Barry Ursick (phonetic spelling) -- I don't see Barry here. Is he--
Victor Gilson, Bridgeton Schools.
Victor, before you start, there are some questions from some of the teachers out there.
The manipulatives have to be punched out right there as you get them?
DR. DeMAURO: They can be prepunched out in advance.
ASSEMBLYMAN ROCCO: They can be punched out in advance. Do the tests have to be scored right there in the classroom?
DR. DeMAURO: No.
ASSEMBLYMAN ROCCO: Can they be taken home?
DR. DeMAURO: No, they're scanned. They're scanned.
ASSEMBLYMAN ROCCO: They're scanned.
DR. DeMAURO: They're sent back and scanned.
UNIDENTIFIED SPEAKER FROM AUDIENCE: Not the speaking.
DR. DeMAURO: What about the speaking?
MR. DOOLAN: Speaking is in the classroom, yes.
ASSEMBLYMAN ROCCO: Speaking -- the rubric, yes.
MR. DOOLAN: That's the only one.
DR. DeMAURO: That's the only one, yes.
ASSEMBLYMAN ROCCO: It has to be done right there in the classroom at that time?
MR. DOOLAN: The speaking?
DR. DeMAURO: Yes.
ASSEMBLYMAN ROCCO: It cannot be taken home?
DR. DeMAURO: No.
ASSEMBLYMAN ROCCO: And yet, while they're doing that they have to find what they're going to be doing with the students at the time. The same holds true with the student who goes first and the student who goes last. In the speaking component, obviously, there is a disadvantage there to some students.
DR. DeMAURO: We're studying that. We're studying order effects. That's what the experiments do. The problem with that is, say you have five prompts -- A, B, C, D, and E. You can vary the prompts within the day so that nobody hears the other child's prompts. Now the standards call for speaking in front of the class-- So you can vary the prompts during the day, but we found that the effect of the overnight was a much greater biasing factor than the effect of the order within the day. We're studying it. We're studying it, but that's what we found.
ASSEMBLYMAN ROCCO: I'd like to see that. And if a teacher calls out sick, does the substitute do it?
DR. DeMAURO: That's up to the district.
ASSEMBLYMAN ROCCO: That would be up to the district. Okay.
H. V I C T O R G I L S O N, Ed.D.: Good afternoon, and thank you for this opportunity. My name is Victor Gilson. I'm Assistant Superintendent of Schools in Bridgeton, New Jersey. I'm here representing the New Jersey Association of School Administrators Curriculum and Instruction Committee and to share with you, specifically, how the implementation of the Elementary School Proficiency Assessment affects my district.
We strongly support rigorous assessment of pupil progress and welcome close monitoring of programs to assure our students of getting the best education possible. The Department of Education has worked diligently and thoughtfully over the years to develop high-quality tests, and those efforts are reflected in the High School Proficiency Test and the Early Warning Test.
Districts across the State have responded to each of these tests by aligning their curriculum, adjusting instructions, and developing assessments which reflect the new standards. In addition, intense staff development was required to help teaching staff members become familiar with the skills and knowledge required of students by the new tests.
As we attempt to raise standards, it is imperative that you recognize the increased responsibility placed on elementary school teachers who are being asked to administer the ESPA and implement each of the core curriculum content standards. It is not simply a case of teachers administering a test. All district staff feel a professional responsibility to prepare our student to meet high standards, and to do that, we need a thorough understanding of the assessment. I'm concerned that we are trying to do too much in too short of a time frame.
The content standards and the ESPA necessitate a change in the way teachers deliver instructions to their pupils. Not only are teachers being asked to adjust their teaching to address new standards in the traditional content areas of visual and performing arts, health and physical education, language arts, mathematics, science, and social studies, they must now teach world languages and cross-content workplace readiness skills. The latter two are additional responsibilities.
In addition, teachers must now assess students in public speaking by using a rubric received less than one month ago. We need sufficient time to decide if the new content areas will be integrated or infused into existing curriculum or will they require separate curriculum guides.
Even though the world languages standard will not be tested until 2001, we must begin preparing next year's fifth-grade and first-grade classes. That means the district must develop curriculum, then train first- and fifth-grade teachers in the teaching of a language. This training must then occur for each grade level in succeeding years. I am concerned that we are attempting too much too soon.
Districts have had sufficient time to develop and align curriculum-- Have not had sufficient time to develop and align curriculum, purchase materials, and in-service staff. Science serves as an illustration. We will be administering the ESPA while State Committees are still working to develop curriculum frameworks. The State's Science Curriculum Frameworks Committee and the Assessment Committee have not communicated because the ESPA is a secure test and all information about it is confidential. Therefore, the alignment of the frameworks and ESPA cannot be at the level it could be if more time was allowed to have the frameworks reflect the assessment.
The math frameworks documents is often held up as the exemplar. The development of the math frameworks involved three to four Department of Education staffers over a three- to four-year period and cost approximately $600,000. By contrast, the Department is attempting to complete the science frameworks with one staffer in a one-year period.
The Science Committee should be applauded for including a performance element in the ESPA. This further intensifies the need for elementary teachers to change their science instruction to reflect the standards, which require an experiential and/or constructivist approach to science instruction. Because this is new, it, too, will require staff development.
While it may be too late to delay administration of the ESPA for 1997, I would like you to consider a few suggestions: One, delay implementation of the performance element of the science test -- pilot it in all districts next year; take steps to be sure the curriculum frameworks document and the ESPA are aligned; three, provide sufficient staff development for teachers -- this is most critical; four, provide direction on the world languages requirement; and five, adhere to a research-based change model.
We are working to develop assessments that serve as models for the entire country. I am hopeful that in our rush to get it done, we remember that there is only one teacher in one room who must do it.
ASSEMBLYMAN ROCCO: Thank you, Victor.
DR. GILSON: Before I close, I would like to-- While much of this appears to be complaining, I don't want it to appear that way. I want to say that validity and reliability can be established using a speaking rubric. Standardized tests are often held up -- whether it's the Metropolitan or any of the five major standardized tests -- as the only way to achieve validity and reliability. That's not true. All standardized tests have a standard deviation.
It has been demonstrated over and over again that with proper training, teachers can administer what appears to be a subjective rubric and achieve or do a better job of achieving validity and reliability on things like public speaking and remain within any standard deviation of the major standardized tests. So it's quite possible to do that.
I would also like to say that it's important that our children know what they're going to be tested on and that we do teach to tests. All of us in our jobs know what we're going to be evaluated on, it's only fair that we know what is on that test, and that teachers know what they need to test.
Scores, I've often-- I've heard repeatedly here today that we want to compare scores. Standardized tests are not developed to compare Bridgeton with Cherry Hill. Since 1964, the Coleman Report (phonetic spelling) indicated -- over 30 years ago-- We know the primary determinate of what a child scores on a test is how much money their mother and father make. So let's stop comparing scores. That's true, I look at it every year in New Jersey. The A districts are at the bottom of the heap, and the other districts are at the top in standardized test scores. Nothing has been done to change that since 1964, and this data is consistent every single year, so let's stop comparing districts. We need to compare ourselves with ourselves, and we need to look for improvement.
The test is a criterion reference test. It is not a norm-referenced test. The standardized tests that I keep hearing about are norm-referenced tests. This is a criterion reference test. They're two different animals.
ASSEMBLYMAN ROCCO: Thank you, Victor.
David Nash, New Jersey Principals--
D A V I D N A S H: Thank you, Mr. Chairman, members of the Committee. Let me, too, express the appreciation of the New Jersey Principals and Supervisors Association for your willingness to hold this hearing today.
This certainly is a very important issue to our members. Our members, as the grassroots leaders of our schools, certainly have a strong interest in making sure that the standards are properly implemented, and when we're moving ahead with developing statewide tests, that we're doing it in a sound manner.
Let me say up front, as you've heard from other testifiers, our members strongly support the idea of academic standards that are very rigorous for all students. We commended the Department for undertaking the idea of developing core curriculum, and we think it's a very worthwhile project.
Our concerns today deal with the manner in which these standards are assessed. We have five basic areas of concern: First, the lack of adequate training dealing with the standards. You've heard this before from other speakers, but we just have not had the kind of training that needs to take place for the sort of complex evaluation we're talking about on public speaking skills. We're going to expect two teachers to be involved in doing uniform assessments, using this uniform statewide rubric, and we just have not seen the kind of training to make sure that that's done in a consistent manner. So we don't think that the two-hour video and the regional sessions we've had are enough training for our members.
And it just leads to a more general point on professional development. We're just seeing now, for the first time, the State of New Jersey looking at a statewide professional development plan, and it's really a backwards process. We should have been looking at a statewide professional development plan before we went ahead with assessing these standards, so that all of our teachers had a chance to have a comprehensive professional development program up front. Unfortunately, that process is going full steam ahead on the accountability side, but we haven't had coordinated statewide professional development to begin the process.
You heard about the strain on districts' resources, and our members have said that over and over again. There is a tremendous strain on district resources being put forth by this testing. You do need to have additional people involved. There will be cost, and that cost has been very effectively documented.
Also, we're going to be asking our members to have fourth-graders spending an hour preparing for this public speaking test, and it's very difficult to ask fourth-graders to spend an hour on any one particular task in an unsupervised manner. So we're concerned about that. We don't think it's a very practical approach that's being used.
The disruption in the educational process is certainly a serious concern to our members. The test, this year, is a very long test. We've heard that, again, from other presenters, and the test is only going to get longer in the future. We're adding new subject areas every year to this testing document, so we'll see in the future many, many days of testing -- more than we have at the current time.
There are also many limitations to the testing process that I think we have to come to grips with. Certainly, when you're looking at writing samples-- As an example, students who are involved in taking a statewide test on writing, you're only going to see a first draft of a student's work. The standards themselves call for a more refined level of skill on the part of students that would come with developing second drafts and analyzing your work and having more detailed work.
In the science area, our standards expect our students to work cooperatively in teams and to have projects worked out in a more cooperative manner. We don't have the ability to do that, to a great extent, on a statewide test. The technology assessment that we'll be doing in the future will certainly call for us to have performance assessments where you'll be utilizing the technology -- something that can't be done very easily on a standardized State test.
And this is only the tip of the iceberg. As we've said, the test will get longer every year. The Department has talked about spreading the testing out over two-year periods in the future -- fourth and fifth, seventh and eighth, tenth and eleventh grades. You could have a tremendous amount of downtime in the educational process.
We believe it makes more sense to find meaningful ways to implement assessment of the standards into the regular educational program. We could begin exploring the idea for portfolio assessment, where a student's work is looked at throughout the course of the school year and across many school years. Not everything needs to be included on a statewide test. There are other ways that we can assess these skills in some areas. Again, we're not opposed to the idea of a test, but we need to look at -- for some of these areas -- alternative assessment strategies.
The difficulty of standardizing the scores is another great concern that we have. We heard for the first time today that we won't be hearing comparisons across districts on public speaking scores. Certainly the sense that we've had up until this point was that we would be standardizing scores in public speaking just like in other areas. It's a tremendously difficult thing to do, especially without the amount of professional development that needs to be done. Writing is another area that is a tremendously difficult area to standardize scores across the State of New Jersey. So we don't know that we have the validity and the reliability to be doing that.
Certainly the failure to address special needs students is a concern that we have. In fact, we just learned today that there are some mistakes in the materials regarding assessment of limited-English-proficient students. The material that was sent out to districts includes in the appendix an explanation that if students are going -- if you have a limited-English-proficient student and you want to exempt that student from the test, according to the appendix for the materials that were given out, you would need to give an alternative assessment to those students. Of course, that's not true, because this is only a pilot test, but that's not explained in the materials that were given out.
Also, if the student decides that they could be exempt from the test, but they want to take the test anyway, according to the written materials, you needed to have written permission from the parent of that limited-English-proficient student so that they would be allowed to take that test. Again, it's a pilot test and that written permission, from what we've just heard today, won't be required for students, but that's in the materials that have been sent out to the districts.
So it's another example just of some of the mistakes and some of the rushing of the process that we have. Our main recommendation at this point is that we have to slow down the process. We know that it looks like, at this point, the fourth-grade pilot test is moving ahead for this year, but there would be no great educational disaster if we had another year of pilot testing this, given all of the concerns and the practical problems in implementing this that we've seen.
So we would urge the Department to slow down, have one more year of piloting this, because when we get to an official test, the stakes get extremely high for all the districts involved. If you have one particular school that doesn't fair well on an official fourth-grade test, eventually, that entire district faces a semitakeover by the State of New Jersey. You would get into a position where, if you have three years of one school not doing well on this test, the Commissioner can come in and take over your district's curriculum, take over your budgeting, transfer personnel across the district, and get directly involved in collective bargaining. We don't think that those are light consequences. They're very serious consequences, and if we're going to have that sort of serious consequence, let's make sure it's an accurate test and a meaningful test.
Again, I thank you for having this hearing today. I do have with me Esther Lee, who is a principal in the Salem School District, has a tremendous amount of expertise on this issue, and would also like to share some of her thoughts on the fourth-grade test.
ASSEMBLYMAN DeSOPO: Thank you much.
ASSEMBLYMAN GARCIA: Thank you.
While John is away, I want to take the prerogative, if Carmine will permit me for a minute. (laughter)
First of all, thank you for your concern for limited-English-proficient students. I'm very thankful, David.
I just want to make an observation, just real quick and off the record before John gets back. (laughter) Just based on what Barbara Michalsky from the Pitman Public Schools presented, about $62 per pupil was being spent on the administration of this test-- In the State of New Jersey we only spend $40 per pupil on technology. So we're spending more to administer -- just to prepare for this test, not even to administer it, because the costs would be higher, than we do on technology. It's just something to think about.
ASSEMBLYMAN DeSOPO: What kind of validity and reliability do you have on that piece of information that you put out there? (laughter)
MR. NASH: If it's the Committee's pleasure, I would like to have Esther Lee--
ASSEMBLYMAN DeSOPO: Bring Esther, yes, please.
ASSEMBLYMAN GARCIA: Can you just give me that -- with the limited-English-proficient students, what you said was different from information or different than what was in the packet? That's important.
MR. NASH: Yes. Sure. Just to go over that again, there's an appendix to the instructional materials that were given on State testing, and the appendix talks about the process for exempting limited-English-proficient students from statewide testing, and it includes in there the ESPA test. Basically, it says that you can exempt a student from the fourth-grade test, but if you do that, you need to have an alternative assessment for that student. It's not explained in those materials.
ASSEMBLYMAN GARCIA: That's the SRA procedure, isn't it?
MR. NASH: That's right. For the eleventh grade, that's the SRA procedure, but it also talks about, for the eighth or the fourth grade, you need an alternative procedure, along the same lines, for these students. Those alternatives haven't been developed for this. It's a pilot test that's being done, and the instructional materials don't explain that for this year, you don't need to do that alternative assessment.
And on the other hand, a student who could be exempted, who decides they want to take the test, according to the instructional materials that are given, you have to have written permission from that child's parent or guardian in order to allow them to take this test. And we've just been told today that for this pilot test that written permission is not required, but that's not included in the materials.
ASSEMBLYMAN GARCIA: Would you please--
If I may, Carmine--
ASSEMBLYMAN DeSOPO: Sure.
ASSEMBLYMAN GARCIA: Can you please respond to both of those issues, please?
DR. DeMAURO: It's not required. It's not required to have parental permission to exempt them from the test.
MR. NASH: Again, that's in the-- It is specifically included in the written materials at this point.
DR. DeMAURO: Yes, we did. That's right. We included it. That's right. We did make an error in providing the eighth- and eleventh-grade policy to the districts as an appendix, and there is information going out now saying, of course, they don't have to. But by administrative code they actually do have to assess the children every year. But they can't use the ESPA for that anyway because it's a pilot test. So they're not free of the exemption requirement, per se. That's in the administrative code. That has nothing to do with our test.
ASSEMBLYMAN GARCIA: And how about next year or eventually when the ESPA replaces, like, the Early Warning Test? Is there going to be a mechanism?
DR. DeMAURO: Yes, we budgeted money to translate the instrument into several languages. If we can keep our budget-- If we can get the money to do that, that's our intention. For those who won't be tested with the ESPA, of course then the district is responsible for assessing them in the way they normally assess them.
ASSEMBLYMAN GARCIA: And on the speaking component, they would be, then-- They could do it-- How would you envision that being done, on the speaking component?
DR. DeMAURO: The real difficulty with speaking -- as for most -- is for those districts that only have ESL only without the full bilingual program. In a full bilingual program, they can be evaluated in their native language. So the issue is to find somebody -- if the child wants to speak in another language -- who can evaluate that speaker--
MR. NASH: I mean, the difficulty is some districts could have a dozen or more languages.
DR. DeMAURO: That's right.
MR. NASH: And obviously you don't have -- dozens of languages -- certified staff in dozens of different languages.
ASSEMBLYMAN GARCIA: So how are we going to address that? I know it's a difficulty. We face that--
DR. DeMAURO: That's why we have-- What we've done in the SRA process is we've made people available who are called native language assessors. It's a problem that every state and every local struggles with, and that is, having statewide materials for every possible second language and having them comparable--
I mean, there are other problems, as well. Translation, for example, changes the constructs. So we have to worry about where you set the standard. You have to worry about the comparability of item difficulties. You have to worry about what they call model slippage and validity. There are lots of problems to address there, and we researched-- With the HSPT, we've done a lot of research on that, and with the EWT, we've done a lot of research on that trying to keep the construct the same. But there is a lot of work that has to go into that before you just put out a test in another language.
MR. NASH: Just on those clarifications, we understand that-- We're hopeful that a note will be sent out -- a memo -- to the county superintendents explaining--
DR. DeMAURO: Yes.
MR. NASH: --those items. It would be helpful if something was sent out directly to each individual superintendent, because sometimes in the shortness of time we have here, the information might not get to every individual district if they don't think to ask.
DR. DeMAURO: That was coming to us today from the Office of Bilingual Education, and Elizabeth said she saw it on the desk just before she left, so we hope we can get that out.
ASSEMBLYMAN GARCIA: Let me just get this, if it's a limited-English proficient student, then they don't have to participate in this test.
DR. DeMAURO: That's right. That's voluntary district practice.
ASSEMBLYMAN GARCIA: And there is no other alternative mechanism for those students at this time?
DR. DeMAURO: No. The State administrative code says that the children's basic skills have to be assessed so that they can know the proper placement for the child in the program, but there is no State test to do that. They're exempt from the State test to do that.
ASSEMBLYMAN GARCIA: So the children -- the LEP students -- then, so far there is no test for them in the fourth grade? Because I know the SRA is for like the eleventh grade.
DR. DeMAURO: Right.
ASSEMBLYMAN GARCIA: There is nothing developed by the Department of Education for all those students who are LEP?
DR. DeMAURO: Not yet.
ASSEMBLYMAN GARCIA: And this test is going to be administered next week, right?
DR. DeMAURO: Well, it's a field test next week.
ASSEMBLYMAN GARCIA: Okay. Thank you.
ASSEMBLYMAN ROCCO: Esther Lee, Salem Schools.
E S T H E R L E E: Thank you for allowing me to speak.
I'll tell you how important this is. This is my very last year; this is my 26th year of education in New Jersey; and the Salem County Administrators are going to be honoring me at a dinner -- it's going to start in 10 minutes in Salem -- so thank you.
ASSEMBLYMAN ROCCO: Oh, my, you should have told me. I would have gotten you on earlier.
ASSEMBLYMAN DeSOPO: We may have to paraphrase.
ASSEMBLYMAN ROCCO: Yes. (laughter)
MS. LEE: I think it's important enough for me to speak to you, and again, thank you for allowing me to address this very issue.
I've been in Salem for 10 and a half years and, yes, we're a DFGA -- I think Dr. Gilson called us the bottom of the heap. Yes, we have a great interest in this test because we're under Level II monitoring. We haven't passed the test that we already have. We've lost money, and so the impact of this test has really hit our school district.
I'm not against higher standards for students, but I'm certainly-- I have a concern in the way the standards and the ESPA was dumped on us. I just want to compare the way the ESPA was given to us and the way we were involved in the math framework project.
Assemblyman Rocco, I gave you a copy of the math framework. That's the original document. We started working on that in 1993 under the auspices of the New Jersey Math Coalition, which included people throughout New Jersey, Rutgers, Rowan, and the Department.
I think that maybe one of the problems that the Department is having I see in Joe Rosenstein's (phonetic spelling) letter that-- In the acknowledgments of that original primary draft -- preliminary draft -- that we had from 1995, he thanked 294 individuals who worked with the math standard. Whereas, I have the original of an attachment that you have, and on here, Commissioner Klagholz responded by saying that -- he thank the 140 members who worked on the whole standards. So you have 140 working on the whole standards, and you have 294 working with the math, so that could be part of your problem.
Again, higher standards are not new. There are national standards. The workplace readiness standards are based on the SCANs report, and the SCANs, by the way, were adopted in Florida in 1991. The New Jersey math standards were adopted in 1996, but they were based on the 1989 National Council of Teachers of Math. So let's compare the way our school implemented the New Jersey math framework and the way that we now had the New Jersey core curriculum content standards dumped in our lap.
For example, if high standards are so important for students in New Jersey -- and I believe that they are and that technology is important -- then we need to have time to implement them correctly. You don't have to look any further--
And, Assemblyman Rocco, I disagree with you about registered holistic scoring, because we have found that when we work with teachers on holistic scoring, they do become between one or two of what the child wrote.
So holistic scoring was implemented 15 years ago by the Department, and take a look at the way the math standards were implemented under the guidance and training of Rutgers and the Math Coalition. The Math Coalition has a good model for us. And if we want to change what's going to happen in the schools, then we must change classroom practice. We have to help teachers change what they do, because unless classroom practice changes, nothing different is going to happen.
The New Jersey math standards, as I said, were built on the math standards that were developed in 1989 by the National Council of Teachers of Math. The current version that was just approved by the State Board last year were the ones that were built on the version that I just handed you from 1993. What we did is -- I gave you a couple of other copies -- from 1993 until three years ago, we were part of the Coalition. Now, in 1993, 7000 copies of that particular document were given out to school districts throughout New Jersey. This included a pilot implementation that involved 60 school districts, including ours, Dr. Michalsky's, and Dr. Gilson's.
For three years, under the guidance of the Math Coalition, six districts from our region, which, again, included Pitman and Bridgeton, met bimonthly to study the standards and the framework one chapter at a time. We practiced, we discussed, we talked about how to assess it, then we went back and we went on to the next standard. Not only did we learn how to implement change in classroom practice, we had input into the frameworks until they were finally adopted by the State Board.
One of the most helpful parts of the math framework that the content standards don't cover is the training -- not just words on paper, but the training that we received over the three-year period in excellence and equity -- that's a very important issue -- keys to success in the classroom, what does a teacher have to do to be successful in changing new practice in the classroom? -- and assessment and planning for change. Because without the assessment part that is ongoing and seamless throughout the year, the teacher really can't tell how the children are doing, and we have to work with teachers in order for them to do that. In the three years, with the math framework project, we were able to do that.
Compare that methodology to what you heard about today, about how the ESPA was dumped on our doorstep for us to implement. And, again, since we're a small district -- we're a district factor group A -- we don't have curriculum coordinators to help the principals and the teachers implement these standards. So I as a building principal worked with the teachers in the framework, and now we're working with them through all the standards.
And although we're testing for the ESPA next week, except for the math -- which we know because we've been involved in the project for three years -- we don't really have any ideas on the major concepts that are going to be assessed and how they're going to be presented. What happened to the concept to test what we teach? Our fourth-grade students are going to be tested in language arts, literacy, and science, and yet, we have yet to see these frameworks. The frameworks are important to teachers and to us as principals because they demonstrate what the standards look like in actual practice.
Now let's look at the fiasco with the speech test. First, here is a copy of the original testing schedule published by the Department. (indicating)
Now, I have the original copy, Assemblyman Rocco. When I cut this out, I meant to use the schedule for teachers. I didn't know I was going to be presenting it at a hearing, but -- you have a copy attached -- there's the original.
Unfortunately, since I didn't know I was going to be at a hearing when I cut that out last year, I didn't put the date on it, but you can see it came from the Department, and if you take a look at it, you can see that the speech and the science -- turn around to the back -- were not scheduled until next year. Now, we heard that the speech was originally scheduled for the spring of 1998, and we heard that it was pushed up to meet the Governor's agenda. We don't know that it's true, but it sounds good to us. Maybe that's why the training was such a fiasco.
Unlike Assemblyman Rocco, I'm not questioning the ability to reach strong validity and reliability on the individual assessment. Because of PSA, I've been working with ETS on the principals' licensure. By the way, that's only a six-hour test, and there are states that are using that to grant a license for a person to practice as a principal.
What we did with that is we looked at the open-ended questions -- and they were defined by a rubric-- But this is where proper training is the key. This is where the difference occurred. At ETS, our group worked on sample responses for three or four hours -- sample-- We looked at the question, we looked at what the possible responses could be, we looked at the rubric, and then we worked on sample questions for maybe up to three hours when we were close together, and then we were able to score the actual samples.
Now take a look at what the Department-- The Department offers us a couple, a few-- There wasn't really enough time to do that, and the video, I have to say -- I'm sorry -- it was very poorly constructed. This was not proper training, and the results are going to reflect this slipshod training process. If New Jersey children-- And I really believe that they deserve -- our kids in Salem, just like the kids in Cherry Hill and Montclair -- high standards, but we need proper training over a period of time.
Now, there is another problem; although-- And I'm surprised that nobody has said this about the academies offering workshops and aligning the curriculum in science and math to the State test. First of all, those academies' workshops only started last month. The workshop -- we attended two, again, because we're in Level II monitoring, we're under a lot of stress because of the test-- We found out that we attended aligning science and aligning math for two days for K to two-- We found that the workshops were not well organized and a lot of times were spent on motivating activities rather than trying to see the relationship of the standards to classroom practice and assessment.
Most distressing to me of all, the presenter -- who, again, is employed by the Department -- wasted the first half hour by lecturing us that New Jersey has too many administrators and that if our district continues to use standardized tests that we should complain to the union. Who can I complain to about two nonproductive days? Well, I'm complaining to you about the lack of help in assessing what we do and in aligning our curriculum to help our students meet success.
I know the Department is so understaffed. Perhaps maybe what the Department should do is use the resources and work directly with school districts to help us implement the standards and leave the testing to people who have experience and expertise in that area.
ASSEMBLYMAN ROCCO: Thank you, Esther, I appreciate it.
MS. LEE: And thank you for allowing me to speak.
ASSEMBLYMAN DeSOPO: Congratulations.
MS. LEE: Oh, thank you.
ASSEMBLYMAN ROCCO: Yes, enjoy your dinner tonight.
MS. LEE: Thank you.
ASSEMBLYMAN ROCCO: Mark Myles, Cherry Hill.
M A R K P. M I L E S, Ed.D.: Thank you, John.
I'd like, also, to thank the Committee for holding these hearings and representatives from the State for being here.
One of the assumptions made in the speaking presentations, as demonstrated in the tape, is that when students prepare their speeches they're less likely to deviate from that and make changes based on what they've heard. This presentation, on my part, is not prepared. So I'm not going to repeat things that other people have said.
In general, I agree with those-- I disagree with the tape analysis; our teachers found it very helpful, and that's from 129 people. My concerns go beyond this administration of the test to the point of the time line that's involved in the getting information back in the districts. One of the things that the test purports to do is identify student progress toward achievement of core content and proficiencies and to replace the standardized assessments that the districts now use. We are not to get results back this year for students. Next year, we are.
But we're not to get that result for an individual until mid-August. I want to notify parents that those students will be getting basic skills and help prior to the beginning of the school year, and I need to do that while the staff is there to analyze the test results and put it into perspective of other things that kids have demonstrated. And I can't do that in August, and the parents won't accept it in September. That's too big of a surprise. I think that time line needs to be addressed.
ASSEMBLYMAN ROCCO: Your time line is-- You said when will the return-- When the schools get the responses?
DR. DeMAURO: In August.
ASSEMBLYMAN ROCCO: In August.
MR. DOOLAN: That's from the field test.
DR. DeMAURO: No, regular administration. There are a couple of difficulties. The open-ended questions on the EWT and the HSPT-- There are like eight open-ended math and four open-meeting reading on HSPT, and five open-ended math and four open-ended reading on the EWT without the essays. It's something like seven million readings. So our priority is to get those tests scored first, so that the kids could have the graduation information they need.
The ESPA scheduling, in fact, is accounting for that whole three-grade testing schedule, which is very difficult, but the remedial model-- I mean, aside from that, philosophically, the State is moving away from remedial model -- the placement of children in remedial classes on the basis of test scores.
DR. MILES: Oh, yes, but parents need to know if students are going to be in need of assistance at some point in time, whether it's a pullout program or inclusionary.
In notifying parents, my intention is to continue to notify parents in the spring prior to the beginning of school in the subsequent year
DR. DeMAURO: That would be difficult.
DR. MILES: In order to do that, that means that I have to place other criteria higher than this high-stakes test.
DR. DeMAURO: First. I don't know about higher.
DR. MILES: Higher. I have to say to a parent, "Based on the information that I've gathered from teachers and other sources, your child should receive some basic skills instruction next year." Then, I get the ESPA results and I have to say, "Yes, the ESPA concurs," or, "No, it doesn't," which kind of places the whole concept of the ESPA in question.
DR. DeMAURO: Well, the ESPA--
DR. MILES: If I can, in fact, identify basic skills students -- for those who need remediation -- prior to receiving the results of the ESPA, then why use them?
DR. DeMAURO: They're moving away from a remedial model. We don't expect that the children are placed in remedial classes on the results of the ESPA. There is no funding for remediation anymore in this State. There is no remedial structure anymore. The idea is to develop a program that addresses the constellation of skills that your students show.
ASSEMBLYMAN ROCCO: I guess the real question is: Is there any chance you're going to move the schedule, or is it locked in?
DR. DeMAURO: Well, the schedule can move. It would be hard for us to field-test some of the suggestions here on a time that's off when we intend to administer the test regularly, because that would give us different results. The fourth-grade teachers told us to make it as late as possible in the fourth-grade year, because they thought that with the knowledge component of it, the children need as much time as possible to have it. That's consistent with our administrative needs in terms of scoring EWT and HSPT, as well.
DR. MILES: I can't refute the idea of the EWT--
DR. DeMAURO: But the scheduling is never stapled down.
DR. MILES: --and the HSPT needing to be done first, but we need the information concerning the fourth-graders in a timely fashion, as well.
DR. DeMAURO: I think maybe there's-- There are ways-- There are some things I think we can look at and we can do that would help get the information timely. I'm just not sure now what all the options are. We'll bounce it off of the people who advise us.
DR. MILES: The interest in having this project done and parents notified as to student performance prior to the beginning of next year -- the next school year -- is very important in our community.
DR. DeMAURO: Right.
DR. MILES: They don't like surprises. To that end, I personally sent a letter out to the parents of fourth-graders in our district telling them about my understanding of the ESPA. And, as pointed out earlier, if it wasn't for Laurie Garcia (phonetic spelling) getting information, I could really validate what I was saying in terms of what the State is trying to accomplish here.
DR. DeMAURO: Right. I sent Laurie that information that she sent to you.
DR. MILES: I know.
ASSEMBLYMAN ROCCO: Okay. Thank you, Mark.
DR. MILES: Thank you.
ASSEMBLYMAN ROCCO: They have the information and your concern.
Marylou Hershmiller, Mount Ephraim. (no response) Okay, that finishes those who have requested-- And now we have a few who didn't get a chance to sign in, so if you will just come forward, give us your name and address -- not address, but your name and school district, and then we'll have that on record.
J A M E S H E T H E R I N G T O N: Thank you, Assemblyman Rocco.
I'm Jim Hetherington. I'm Superintendent of Schools in Gloucester City. I know you've been here long; it's been a long time. I just want to thank the woman who is sitting along side of me -- who is one of my fourth-grade teachers -- who said, "Is there going to be a hearing? May I come and say a few things?"
Jay and Gerry, I think you need to hear some frustrations from the trenches.
This is Elaine Stephenson.
ASSEMBLYMAN ROCCO: Elaine, how are you?
E L A I N E S T E P H E N S O N: Hi. I'm glad for this opportunity, and I'm thankful that you have afforded this to us.
Ever since the teachers in our district -- we have six fourth-grade regular classrooms, and then we have three special ed that teach children who are identified as fourth-graders in their classroom-- Everybody has been upset, just like you're hearing already. And they've been saying, "Why doesn't somebody listen to us? Somebody doesn't care what we have to say," and so forth. So they listed some concerns, and I was very grateful to hear that they've already been expressed for the most part -- not 100 percent, but for the most part they've been -- and the teachers will be glad to hear this when I go back tomorrow.
I know more than most of the teachers simply because, being a little bit older than the others, my family has grown up, and I can take the extra time. But the teachers who are younger and less experienced or have a little less time to give to this are really feeling an awful lot of anxiety and frustration.
My principal, first of all, wanted me to make sure that I clarified about the fact that it was -- whether or not it was a diagnostic test. Now, I'm a little bit hearing impaired so I didn't feel I was given a clear answer on that. I see it as evaluative of the district meeting the standards, but I have not seen the test so I don't know. How is it diagnostic, if it is?
DR. DeMAURO: It distributes the children in three levels, and it distributes them according to accomplishment of knowledge or skills that they need, to have the knowledge for fifth to eighth grade. Those three levels are those who have both the knowledge and skills necessary. Then the separate level is those who don't have either, and the middle level is conceptualized as those who have the skills but haven't yet acquired the knowledge. Of course, it could be the other way around, too, but the way we built the test, we expected that as the middle level because the skills are a prerequisite to knowledge acquisition.
MS. STEPHENSON: I'd like to--
DR. DeMAURO: So it's broadly diagnostic in that sense. It would be--
MS. STEPHENSON: Broadly.
DR. DeMAURO: --more diagnostic if--
MS. STEPHENSON: It's not specifically diagnostic. It's broadly diagnostic.
DR. DeMAURO: Well, it's diagnostic to that extent -- subject area to subject area. If you wanted more diagnosis, you would have to add more items for greater reliability, pinpointing kids in different points of the distribution.
ASSEMBLYMAN GARCIA: I know why the teachers are confused.
MS. STEPHENSON: In other words, we really can't substitute this for a diagnostic test for the purposes that we need it in our district?
DR. DeMAURO: Without knowing the purposes you need--
MS. STEPHENSON: You wouldn't be able to answer that. But I don't see it as that.
DR. DeMAURO: A diagnostic test for the sake of accomplishment of the content standards.
MS. STEPHENSON: Okay. And, also, did you say that we will only get results if we request them?
DR. DeMAURO: There is no intention now to give results on the basis of field tests, because we don't know which of those items will work and which won't.
MS. STEPHENSON: Okay. Also, I would like to know how you can expect to get any valid information from this test when the schools have not had a chance to get -- and they're not even required yet to have -- their curriculums in lines with the State and national standards. I've been fortunate to be participating in Project Smart in our school district, and I'm still learning the full understanding of each one of those standards. They are not easy to identify.
I don't know if you gentlemen have taken the time or been able to take the time to read them.
ASSEMBLYMAN ROCCO: Every single one of them.
MS. STEPHENSON: But it is not easy to understand those standards and, on top of that, to be able to find the materials to address each of those standards. The teachers have been doing that in spite of the fact that we haven't even had a chance to address the curriculum to the standards yet. So I don't see how you can get valid information. We have not had an opportunity to do that. We're struggling with it at this moment.
The children have not been exposed to the instruction that is necessary in fourth grade alone, not to mention the fact that K to three hasn't been able to address these things at this point. So your information cannot possibly be valid. That's the thing that I'm really concerned about. You can't evaluate something that hasn't taken place.
DR. DeMAURO: Nor do we intend to.
MS. STEPHENSON: I know, but I'd like somebody to take the time to write me an explanation as to how you are going to be able to use this information and what validity it can have for you when nothing has been addressed; nothing is in place yet. It's very frustrating as an educator.
I'm also very concerned about the fact that this security business makes educators feel that you have absolutely no confidence in us. Even if we saw this material a month ago or six weeks ago, we couldn't give these children a crash course in this. So there is no way we're going to be teaching these children to the test or throwing off the validity that I don't believe exists anyway at this point.
And the idea with the manipulatives-- This little discrepancy on the ruler that has been mentioned-- I'm not aware of it, I haven't looked at it; however, is the purpose to evaluate what children have learned--
DR. DeMAURO: No.
MS. STEPHENSON: --or trip us up for something we overlooked as educators?
DR. DeMAURO: I'm sorry, the purpose is for neither of those. It's to build an item pool. It's the quality of the items we're looking at. We're not evaluating your program or your implementation of standards this year. It's a field test to build an item pool.
ASSEMBLYMAN GARCIA: And once--
Excuse me, if I may, through you, Mr. Chairman.
After this year, after the field test is conducted, what is it intended to measure?
DR. DeMAURO: Well, again, it's broadly diagnostic in the sense of whether the students have achieved the K-to-four standards and are now prepared for the five-to-eight standards. That's what it's designed to measure.
ASSEMBLYMAN GARCIA: So it's going to measure for the students how much they have absorbed in the area -- in the content areas that are prescribed?
DR. DeMAURO: And the skills they have that prepare them for that, yes.
MS. STEPHENSON: Which we haven't addressed yet.
ASSEMBLYMAN GARCIA: Right.
MS. STEPHENSON: Thank you.
MR. DOOLAN: But there are many districts that have begun to do that process. We hear that all the time.
MS. STEPHENSON: We're working on it because we care so very much.
MR. DOOLAN: Good.
MS. STEPHENSON: However--
ASSEMBLYMAN GARCIA: But they were only devised like six months ago.
DR. DeMAURO: No, no, the standards were passed in May of 1996. This is-- It's a six-year period between when the standards were passed--
ASSEMBLYMAN GARCIA: All right, a year ago.
DR. DeMAURO: --and when the last field is given. It's a six-year period. That's why the first tests are in the basic skills areas. There is a six-year period between standards passing and the last field test.
MS. STEPHENSON: I just still cannot see how this can be a valid test when children and teachers-- Children have not had an opportunity to be exposed to what they need to be exposed to and educators have not been able to align their teaching to what the children need to have provided.
DR. DeMAURO: If the purpose of the test were to evaluate your program, you'd be absolutely right. That's not the purpose of the field test.
MS. STEPHENSON: What is the purpose of this then? What is the purpose of the test?
DR. DeMAURO: To build an item pool. The purpose of the field test is to build an item pool. The purpose of the test is to distribute kids diagnostically as to those who have acquired the knowledge and skills that the-- The way the standard words it is, what they know and what they should be able to do. Those who haven't acquired either and those who have acquired the skills which prepare them for acquiring the knowledge but haven't yet acquired the knowledge-- There are three levels. That's the purpose of the test.
The purpose of the field test is not to evaluate. The purpose of the field test is to see which items are best correlated to the constructs, which items are not biased -- because we do a bias analysis, and we get rid of all the items that are biased -- which items are in the right level of difficulty and have the right item characteristics so that they can go into a field test that would be sensitive to evaluation in two or three years.
If you-- You were absolutely right. If the purpose of this test were to evaluate the district implementation of standards, it wouldn't be valid. That's not the purpose of this field test.
MS. STEPHENSON: And next year it will be to evaluate the children's level of performance within the standards?
DR. DeMAURO: Yes. We'll have a good item pool then.
MS. STEPHENSON: Even though they will not have had four full years of implementation into the standards?
DR. DeMAURO: Next year, they will have had two years since the passage of the standards. One of the--
MS. STEPHENSON: Passing of it, but you haven't even required the schools to have their curriculums in line. This takes a lot of time. Teachers have not had the proper training. I told you, I have the advantage of Project Smart that has been explaining a lot of this and giving us materials to address these things. Most of the teachers have not had this opportunity. There is a handful that have had it. So you're expecting an awful lot, and you're putting an awful lot of stress on teachers -- on children, because of the fact that we're stressed, there is no way that they can help but to not feel the anxiety that we're feeling at this point.
And, like I said, as far as this security-- It has put-- I mean everybody is starting -- feeling, probably unneeded; however, you know, you can't see this-- And then you get a little upset about the fact that this is all secret, and they shouldn't have seen them, but I just read in the manual -- and I can read the paragraph -- it says that the teachers are allowed to have those ahead of time to get them ready and into the bags for the children.
And, as far as the grids, we need time to have those in our hands to get the grids ready for the children because -- I'll tell you, I do the Californias for my children, and, even at that, I can make mistakes with those. So therefore, when you're putting these constraints-- You're doing it to our administrators, our coordinators, our test coordinators, who have to make sure that we don't have this material too long, and so forth. I think that you're going overboard with this.
DR. DeMAURO: Well, we've had several instances where people have been-- Security is really a bias issue, because nobody should have unfair advantage. And we've had instances where people have had an unfair advantage, and all districts aren't subject to that. So the nature of standardized testing allows for security.
Secondly, we need security. The basic equity issue is that every form of the test is comparable in difficulty to every other form in the test. So if there is a difference in the scores of those two forms -- as best as we try to make them parallel, sometimes we slip by half a point or something-- If there is a difference, we have to either be able to attribute it to the test being harder or the class of children who took the test being more skilled. The only way you can make one attribution is either hold the children constant or hold the set of items constant; that's what equating does. To have a set of items constant means you have to have security. That's the most basic equity issue.
MS. STEPHENSON: I can understand it to a certain point, but you're making it to a point that it's difficult.
ASSEMBLYMAN ROCCO: All right. Let's not debate security here. Let's not debate security. Do you have any other points?
MS. STEPHENSON: Yes. Also, in it the standards are stressing -- and it was brought up a little bit ago -- the cooperative learning aspect of education, which is really valuable; however, in the testing, the separation of the children is important and it's mentioned in the manual. So you're going to teach a child to work with someone to help them, but to boost their -- or to brainstorm, or whatever, to get more creative or thinking going on--
MR. HETHERINGTON: The speaking component of the test?
MS. STEPHENSON: No, this is in-- In speaking, they can't cooperate. They can cooperate in any part of the test. But what I'm saying is, you're encouraging cooperative learning, and, basically, the standards are all encouraging cooperative learning and those kinds of things -- where it's hands on, student involvement, group involvement -- but then you test them in isolation and separation, and it's important that these children, according to the test manual, be separated. Okay? So we have a little conflict here between what your principles in education are and then what you're doing when you're testing -- is a concern here for me.
Also, there are other problems that the administrators are going to have with rescheduling, which has already been mentioned. But the quietness of the testing atmosphere with bells supposedly not on, but we're the only classes in the building that are going to be testing and the other children still have to be in their regular learning environment. So they're just some of the things I thought needed to be addressed.
The biggest thing is that it's very frustrating to be evaluating something that has just not been put in place, and the validity of-- I see that you're going to see how valid the questions are, but that doesn't mean that you're going to get that if the children can't do it, because it isn't their fault that they don't know it.
DR. DeMAURO: Well, we're not evaluating the children.
MS. STEPHENSON: And when a child reads something they say, "I don't understand." It's bad enough when I explain to them on the California test, "They're designed that way. Don't get upset if something is not easy for you or you don't know the answer to that." But if just about everything on this test is a frustration for the child, that's going to be a concern for me.
ASSEMBLYMAN ROCCO: Okay. I think you've driven your message home, pretty well.
MS. STEPHENSON: Thank you. And thank you for the opportunity. I especially think the teachers need to know that somebody is caring about what's going on with this.
ASSEMBLYMAN ROCCO: Well, that's what the Committee is here for, and we're happy to do it.
MS. STEPHENSON: Thank you.
ASSEMBLYMAN ROCCO: Thank you very much for your testimony.
Are there any other speakers at this time? (no response)
That being the case, Jay -- really -- and Gerry, we thank you very much sitting through the entire thing, coming up with your perspective. I think you have some idea as to what the main concerns are out there.
MR. DOOLAN: We certainly do.
ASSEMBLYMAN ROCCO: I think it's going to-- I think the points that were raised today are pretty much those that are pretty constant throughout the region.
MR. DOOLAN: I agree.
ASSEMBLYMAN ROCCO: And, Rudy, thanks for coming all the way down here.
ASSEMBLYMAN GARCIA: Thank you, Mr.
ASSEMBLYMAN ROCCO: Thank you, everyone, for coming. The hearing is adjourned.