Tired, Old, Angry, and Worn

I spent the better part of the day today dealing with unhappy people.   The morning started with an unhappy student who didn’t understand why because she couldn’t effectively use web publishing software, we couldn’t just do the job her department hired her as a GA to do for free.

She then preceded to write me a lengthy email describing to me all the things that were wrong with the way I did my job.   This is after I had personally, or someone in my group had, spent a not small number of hours trying to help her impress the people she was trying to impress, and make her a success at the project she was trying to pull off.

Flash forward to just before lunch, and I exchanged emails with an unpleasant coworker who has a nasty habit of focusing on trivial things like background colors on web pages during finals week when we’re trying to both close down Spring, and get the Summer sessions in startup.

After lunch, I got to read through another in an endless string of increasingly restrictive and unnecessary travel policies designed to make sure that if you’re ever dumb enough to travel on the State once, you’ll never do it again.

After digesting that gem (a favorite of mine given that I have a long and sordid history with travel reimbursements) I fought with a system issue that I didn’t completely understand – made somehow worse when it suddenly started working and I didn’t understand why.

Just before leaving, my phone rang and it was a very sweet, but clearly frustrated person on the other end of the line who was trying to accomplish what should have been a simple task – but do to an uncooperative system just wasn’t able.   She and I, both at the end of our patience for the day, decided to call it a day and try it again in the morning.

I left work for an appointment, and after an hour of being offline, opened my email to 16 unread items related to performance issues that are sure to spawn necessary but painfully derailing discussions tomorrow.

Finally, I came home to an IRS audit notice.   It was at that point that I sat down to rant out my frustration and say what I’d be longing to say all day but couldn’t find the appropriate time.

Fuck today.  Fuck today hard.  I hope it dies.

Changing the Future through Parenting

Yesterday afternoon, I was discussing the horrifying and brutal killing of Shaima Alawadi with my wife Laura, when our nine year old daughter Sydney came out on the porch and overheard us talking.

Rather than immediately stop the conversation because I thought it inappropriate for her age, we let her listen for a bit and form some questions about what she was hearing.   She wanted to understand why this woman was killed.   Trying to explain to her that there was really no other explanation other than “sometimes people are afraid of people who are different than they are, and they do stupid, evil things because of that fear and hate” didn’t completely resonate with her.

That explanation, even to a nine year old, seems so insufficient.   It led to a more important lesson that I seized the opportunity to impart to her.  Based on the way she listened and seemed to internalize, I know she thought seriously about what I was saying.

“Sydney, what you need to know is that it’s not ok to hate because someone is different than you.  Not because they have a different color skin.  Not because they have a different religion.  Not because they talk differently, or act differently, or look or sound differently.   But it’s not just enough that you yourself don’t hate others because of these things.  It is your responsibility to stand up for others when you see someone attacking them for any of these reasons.  One of the privileges you have being, albeit momentarily, in the majority in our country, is to stand up for what is right, and make sure others different than you don’t have to face this kind of hate.”

I hope if enough parents of young children try to impart a similar lesson their generation will never have to read a horrible “Shaima Alawadi” story.

Haters gonna hate

Yesterday, the singer/guitarist and primary songwriter of the band Fall Out Boy, Patrick Stump, wrote a lengthy post on his blog  about the frustrations he’s been dealing with personally and emotionally since the band fell out of favor a few years ago.   I’ve shared it with a few people, because though the average persons situation differs from Stumps in a lot of ways, some of the basic frustrations he talks about resonate with me.

Some of the things Stump says in his post that jumped out at me:   When talking about going out on tour in support of his solo album and encountering fans who were less than appreciative of his performances – “What I wasn’t prepared for was the fervor of the hate from people who were ostensibly my own supporters (or at least supporters of something I had been part of). The barrage of “We liked you better fat,” the threatening letters to my home, the kids that paid for tickets to my solo shows to tell me how much I sucked without Fall Out Boy, that wasn’t something I suppose I was or ever will be ready for.  That’s dedication. That’s real palpable anger.”      On talking about trying to please past fans with new work – “there will still be 10-20 percent of the audience there to tell me how shitty whatever it is I’m doing is and how much better the thing I used to do was.”

I can sympathize with his frustrations.   I realize I run the risk of sounding like an increasingly old, angry, codger (which I suppose to some extent I’m becoming), but it seems like the loudest voices of dissent and dissatisfaction these days come from people who have personally accomplished the least.

They can’t show any demonstrated success in things they’ve created, contributed to, or thought of – but they’re the first to jump up to shout down someone else’s attempts to try something new.    It’s frustrating, it’s disheartening, it makes everything less fun, and it makes you want to just turn your back on them and walk away sometimes.

I expressed it to a friend of mine in this way:  “You can tell them ‘I created this according to your specifications, am delivering to you at the date and time you requested, and I’m going to blow you while you take receipt of it” and they’d still tell you ‘You suck’.

Bottom line – if you’re not willing to do it better yourself, don’t criticize someone else.


Things I Am Looking For

Forgive me the random stream of consciousness brain dump of things I’m looking for – but if you are interested in talking about any of these things in more detail, let me know.

  1. HBO Go – I’m looking for someone who pays for an HBO subscription, but doesn’t use the HBO Go service (a free part of your service) and wants to let me use the digital version of their service in exchange for part of your monthly bill.
  2. iPod Touch – I’m looking for a recent iPod touch model for Bo. I’d be willing to buy one at a reasonable used price, or trade a PS3 for it.
  3. A bike tune up – a good recommendation on getting a bike tuned up and in riding shape for the spring. I’ve used Jeff’s before, and honestly wasn’t super happy (the gearing was wonky immediately after the tuneup) so I’m looking for alternatives.
  4. Thanks in advance for any ideas or help you share.

IMS Enterprise 1.1 Vista XML Integration, Blackboard Learn 9, and Processing Enrollment Changes

UPDATE: I apologize for the missing images.  They were Blogger hosted originally and seem to have been corrupted/went missing in the transition to Google+/Picasa.   You should still be able to get the gist of the solution without them, but if you still have problems, please contact me directly for more information.

Having just worked through a migration from Vista 8 to Blackboard’s Learn 9.1  LMS platform, there were some challenges around translating the formerly “live” integration achieved from Banner to to Vista via the LMB to the new XML Feed file integration provided by Blackboard’s Learn 9.1 platform.

I should start this post with the caveat that LMS Administration is a fairly new pursuit to me, by virtue of some staffing transitions at Marshall University – so it’s possible (likely even) that if you’ve been doing LMS administration for some time, much of what I describe as I walk through how we achieved our integration will be second nature to you.   It’s also likely that you’ll laugh at a few of my silly mistakes – but I’ll discuss each of them anyway, in the hope that someone else in my shoes can benefit from the experience.

Our Environment

In the interest of background to help you determine if this post is right for you – as well as in an effort to hit on some of the keywords most likely to bring the right information seekers to this post, I’ll talk through our previous and current environments.

The old environment

Blackboard/WebCT Vista 8 Enterprise
Banner SIS sending person and membership record updates via the Luminus Message Broker

The new environment

Blackboard Learn 9.1 SP8
Banner SIS providing XML exports of person and membership records
Export files posted and processed by Learn 9.1 via the IMS Enterprise 1.1 Vista XML Feed File integration

Our Situation

The previous semester, we had run a dual environment where Learn 9.1 was our “pilot” LMS with a limited number of live course offerings, while Vista remained our core LMS.

We had always planned to migrate all of our live courses to Learn 9.1 for the new spring semester, and
with staff transitions happening, and the reality that someone with little knowledge or history of the LMS was going to have to quickly get up to speed, it made more sense to push forward on one system than to try to put on the brakes on our planned rollout and continue to support both systems.

Pushing forward meant quickly figuring out how to make integration between the SIS and the new Learn 9.1 system as seamless as possible with the loss of the “live” LMB integration.   It also meant that there would be a lot of trial and error, and learning from silly mistakes along the way.

Making Integration Work

Let’s walk through the steps we took to get our integration working.   Along the way, where we encountered problems, made mistakes, or had to work around an issue, I’ll call that out along with any additional information that might prove helpful to someone else in a similar situation.

Step One – Setting up a semester data source

The first step I took was to set up a new datasource in the Learn 9.1 admin to handle the new semester course and enrollment data.   By assigning a new data source key to each term, data can be manipulated in sets that correspond to logical term dates.   This way, if a change needs to occur that impacts courses, courses can be manipulated by term without affecting other data in the database.   Additionally, setting up a new data source key for each term eases reporting later, when you want to pull out information about a specific time period.

To create a new data source, log in as a system administrator, and click on the Data Integration building block link:

Next, click the “Data Sources” Link:

Click the button labeled “Create Data Source” at the top of the page:

From here, you’ll be presented with a “Data Source Properties” dialog.    Give your data source a key that makes sense to you, and a similarly descriptive bit of text in the description box.   Since the data source I was creating was for the “2nd” term of our 2012 academic year, the key I chose was 201202.
Step Two – Creating an SIS Integration for your new data source

Once you have your new data source created, you need to either tie an existing integration to the new data source, or else create a new integration for the newly created data source to feed it data.   For simplicities sake, I created a new SIS Integration to match the newly created data source.  To do this, follow the same steps as you would when creating a data source – but this time rather than clicking on the “Data Sources” link on the Data Integration Page, click on the “Student Information Systems Integrations” link.
On this page, clicking the “Create Integration” button at the top of the page will expand a drop down menu showing you your available integration choices.

Since we are using banner as our SIS, and we know that it’s producing IMS Enterprise 1.1 – Vista compliant XML files when we run our user and enrollment queries, we selected “IMS Enterprise 1.1 – Vista” when creating the new integration.

Important note:  You can have more than one integration tied to a given data source.   In fact, there are some cases where you may need to perform an administrative level action on data in your database that will require you to create a flat file integration.   I ran in to two of these cases myself when I needed to change the data source an existing user record was associated with.    The important thing to understand is that it’s not a big deal to have more than one integration set up for a given data source at the same time.

Once you click on the type of integration you want to create, you are taken to the configuration page for that type of integration.   The information that you need is going to vary depending on the type of integration that you’re setting up.    Since, in our case, we were focusing on IMS Enterprise 1.1 – Vista, that’s what the screenshots I’m providing will be of.

This type of integration will present you with three primary configuration sections:  Integration Properties, Data Support, and Advanced Configuration.   Let’s break each of these sections down and talk about key components.

Integration Properties

Because I quickly understood that each semester I was going to want a new data source, I initially thought that I would need to do the same thing with the integrations I was creating.    So, naturally, I named my integration 201202 just like my data source.    As I later understood, you can take an integration and tie it to another data source at any time, so what you call your integration doesn’t need to reflect your database naming structure.   If I were doing it again from scratch, I’d probably just give the integration a name reflective of the type of integration being created “Banner-IMS-Vista” for example.
The “Shared Username” you see here is a unique identifier tied to this particular integration that you’ll use as a username if you want to take advantage of the web services end points provided when you create an integration to auto-post your XML files.
The “Shared Password” field is where you can supply your own unique password to be used when calling these integration end points as web services.
“Integration Status” is self explanatory – as is “Log Verbosity”.

Data Support

In the data support section, choose the radio button labeled “Use the same Learn Data Source for all new inbound data” and choose the data source you created in the previous step from the drop down box provided.
You should be able to leave all the other toggles at their defaults.   I was, but your mileage may vary, so be sure to test with your data before pushing live.

Advanced Configuration

The “Advanced Configuration” section is where you’ll define how this integration handles the processing of the XML that you’ll be importing.   What you set these to will depend on your particular institutional business rules – but in general, it is better to use “Disable” than “Purge” in your deletes column if there’s any doubt as to whether you’ll want to keep the data around.
Once you’re happy with the configuration you’ve set up for each type of data you’ll be sending over, click the “Submit” button at the bottom of the page to create your new integration.
Step Three – Test your newly created data source and integration

After you’ve set everything up, you’re going to want to test your newly created integration by grabbing some person or membership records from your SIS in XML format.   Until you’re sure things are working the way you want, you might want to just create some sample records from a subset of your data.    It’s also helpful to remember that the integration is going to process an enrollment, generally, in this order:
1. Load a course
2. Load a section
3. Load a faculty member person record
4. Load a faculty member enrollment record to associate them as instructor for that section
5. Load a student person record
6. Load a student enrollment record enrolling them as a student in a given course
Remembering this general flow is helpful when deciding how to load your data tests so you don’t end up getting records when the integration tries to enroll a student in a course that doesn’t already exist, for example.
Test By Uploading a Feed File

To test this type of integration, again click on “Student Information Systems Integrations” in the Data Integrations Building blog section of the admin.    Find the integration that you created, and click on the drop down link to the right of the integration name.
The two most useful links in this menu during your testing are “Upload Feed File” where you’ll actually upload the XML files that you want this integration to process, and “View Log”, where you’ll go to check and see if the processing worked as you expected.
After you’ve uploaded your feed file, come back and click the “View Log” links to see if there’s anything unexpected happening.   If there is, you should see errors appear that are reasonably easy to troubleshoot (usually improperly formatted XML).     Assuming no errors, go ahead and take the next step of verifying that the records are showing up in the admin UI after processing by doing some searches for users, courses or enrollments.
Step Four – Automating your Feed File Uploads

Assuming that everything is working as expected, you’ll want a way to automate the upload process so that you won’t need to have someone visit the site each day and manually upload these files.   This is where the Shared Username, and Shared Password that were configurable when creating your integration come in to play.
To post XML feed files to the integration you’ve created automatically, you need three pieces of information.   You’ll need a username and password (which we’ve already covered), and you’ll need to know the endpoint URL that you want to post to.    The endpoint URL is provided in that same drop down menu used to upload the feed files.
Grab that endpoint URL, and set up a curl, python or similar script to post the contents of your feed XML to the endpoint using your shared username and password for that integration.
In our case, we used a curl post that was set up to run at periodic intervals throughout the day.   The duration you’ll need to wait between posts is dependent on how often you need your data to refresh.   Since we have username and password data for our students included in the feed posts, and since students can and do change those pins periodically throughout any given day, we chose to refresh our data on a one hour cycle.
Handling Unexpected Issues with Integration
There were a couple of places where a lot of time was spent banging my head against my desk because things were happening (or more often were not happening) as I expected once the automated feed file posting was set up.    Let me step through each of these.
Problem #1 – Many users calling our service desk unable to login

After the integration was set up and working, and had been tested as much as I could test it prior to going live, I felt confident that all systems were go and things should work as expected.   When users started getting locked out and pin/password resets weren’t having any effect, I had to investigate why.
As it turns out, in addition to the initial integration configuration screen that you see when first setting up the integration, there is a bit more detailed configuration screen available that allows you access to field mappings so that you can match data fields in your SIS to the fields that Learn 9.1 is expecting and understands.
To get to this advanced configuration, you again click on the “Student Information Systems Integrations” link in the admin, click the drop down link next to the integration in question, but this time click on “Advanced Configuration” in the third portion of the drop down menu.
Doing that will bring you to this screen, where you can further drill down in to each specific object type.
Since in the case of user pin resets not working we’re dealing with user objects, I can click the drop down next to “Users” to access the field mappings configuration for this object type.
On the field mapping screen, the first thing I noted was that the check box under “Change on Update?” was not checked for User Passwords.   Bingo – that must be it.   I happily checked that, and went on my way, thinking that passwords would now sync.
Sadly, this didn’t correct the problem.   Because our passwords are encrypted, I need to come back to this screen and also check the “Password Encryption Type” change on update checkbox before password resets began syncing up.
Problem #2 – Processing Drops, Changes in Enrollment Availability

The next major issue was handling users who would drop a course.   The documentation provided around integration leads you to believe that if you enroll a user in a course, and then later rerun membership data where that user isn’t tied to that course, the enrollment record for that user will be disabled.
This isn’t how it works.   In fact, users could drop a course, and have no further reflection fo their user being a member in a course in future XML feed pushes, but this had no effect on their membership in the course in Learn.
As it turns out, what we’re dealing with is another field mapping issue, this time around the “Enrollment” object, and the “Row Status” change on update setting.   Row Status is what controls whether a particular user record is set to available or unavailable for a given course.
This value *should* key off of the <status></status> value in the membership XML.   According to documentation, if the status is set to 1 in the XML, the user is set to “available” for the course in question. Conversely, if the status value is set to 0 in the XML, the user is supposed to be set to “unavailable”.
In our case, setting the status value to 0 (which is what happens in Banner when a user drops a course) wasn’t having any impact on the user availability in Learn.
In order to make this work, you actually need to modify the “Source Field” section of the “Row Status” in the enrollment field mappings to use a custom script.

The custom script essentially tells learn how to properly interpet the status values coming in from Banner in the XML so that users can be marked as unavailable when they drop a course.

The script that we used to make this work (again, this goes in the “Source Field” of the Row Status row in the Enrollment object field mappings) was:

var rs = true;

if ( data.rolesToAddOrUpdate.get(0).status == ‘Inactive’ ) {

rs = false;



This allowed drops to process correctly.

Moving Forward

I expect to have additional issues as we move through the semester, and I’ll come back and update periodically.   If you have any useful information or insight based on experience at your own school, please share in the comments.