Avoid Idling

The advice to 'avoid Idling' may sound rather puritanical and a bit on the workaholic side, but I pay it special heed since it comes from Arnošt Löbel, Sr. Principal Engineer in the Revit development team, our resident Revit Idling expert, who already provided many (or almost all?) important insights in this and other related areas:

• Transaction responsibility
• Regeneration option best practices
• Asynchronous API calls and Idling
• AutoJoinElements
• Closing the active document and why not to
• Modeless door lister flaws
• Warning addendum on switching views
• External application attributes
• Extensible storage vendor id
• Modeless best practices
• Warning on using undocumented ElementId relationships

Arnošt also presented a class on the Idling event at Autodesk University 2011, CP5381, on asynchronous interactions and managing modeless UI, which would be well worth a blog post all on its own.

Arnošt clarifies that he is absolutely not against the Idling event, in general. The arguments below are only about using or not using Idling in one specific pattern – reacting to modifications of the model when both external well as internal data (in the model) could be changed in effect of the initial modification. In such situations he does not recommend utilizing the Idling event, because it does not support the pattern well.

There are, of course, situations where Idling is very much in place and some scenarios when it is quite necessary, such as interaction with modeless dialogues and multithreaded add-ins.

With that out of the way, here is an interesting conversation with a number of new important hints between Arnošt and Joe Offord of Enclos on the pros and cons of

• DMU, the dynamic model update mechanism,
• DCE, the DocumentChanged event, and
• IE, the Idling event.

Question: I have a process that is monitoring certain elements in a model via the DocumentChanged event. If I decide I need to perform a transaction on some of those elements I am storing the Document object via DocumentChangedEventArgs.GetDocument, along with some ElementIds, in a customized list that the Idling Event will process later. I then subscribe to the Idling Event.

When the Idling Event triggers I retrieve the same document object from my list and perform transactions with it.

Is this a safe way of transferring elements between the DocumentChanged and Idling Events? After attending Arnošt's class at AU I'm not 100% this transfer is "thread-safe". If this isn't safe what would be a better way to define document-specific tasks for the Idling event to process? I understand the DocumentChanged event does not me to make changes inside that event.

Answer: You present a design pattern that is not easy to solve or give a simple resolution for. First of all, we need to know whether your approach is correct.

We have two patterns for applications to react model changes:

1. DMU – used when model changes trigger other changes in the same model
   1. The document is modifiable, naturally
   2. The changes are all within the document (the actions as well as the reactions)
   3. You do not need to know about Undo and Redo because Revit handles it automatically
2. DocumentChanged – used when model changes trigger changes to external data
   1. The document is not modifiable
   2. The applied changes are made to external or non-model data
   3. You need to know about Undo and Redo, because Revit does not manage the external data

You seem to be merging those two patterns. As mentioned above, we need to determine whether this is necessary at all.

My recommendations and answers:

• I, personally, do not recommend the DocumentChanged + Idling event, because – quite simply, it is not what Idling is for.
• Is it a safe solution? Well, more appropriate question would be: Can it be implemented safely? The answer would be a yes; it can be implemented safely, but one needs to take into an account a lot of other things besides looking for DocumentChanged. When you have a gap between two calls into the API, you need to be aware of everything that can happen between those two times:
  • Document may be closed
  • Document may be saved, saved as, synced to central, reloaded latest, etc.
• Besides all those other things that may happen between two events, it is indeed safe to rely on data gathered during the last DocumentChanged until the next Idling, because it is guaranteed that no other change has been done to the model (except the work-sharing operations mentioned above). To be absolutely certain about elements, always use Unique Ids over regular Ids. Unique Ids are guaranteed to works across work-sharing operations, regular Ids are not.
• My preference is to do without the Idling event. I think that if I were challenged by the problem your customer is facing, I would try to implement it with DMU and DCh event only:
  • I would modify my external data on DCh even only
  • I would modify the model during DMU only
  • I would give a lot of thoughts to synchronize those two well

If I know more about the customer's project, I may be able to give more specific advices. Why do they need transactions if their tool is for is maintaining a live link between Revit elements and an external database, for example?

Response: Thank you for the feedback. Let me try and further explain the problem I'm facing.

In Revit, I'm using a specific generic annotation family to track and quantify my objects in the model. We have an external database that stores all their properties. The annotation family has custom instance text parameters. In the external database, instances are tracked by their name and their host View name. I do not want to go through reasons why we use an annotation symbol rather than modelling each object but I can assure you this is the most efficient method for us.

Via the API I've created an interface that allows the user to pick an object from the database and have it drawn on a Revit view as an AnnotationSymbol. The parameters are automatically filled out via the API. I then have a routine that counts up all the object AnnotationSymbol instances in a particular view and manually draws them in a schedule using lines and text (as a Group: this is required because Revit will not create Schedules from Generic Annotations). At the same time, all the instances of that object are uploaded to the external database.

Now, my goal is to have any changes to the model automatically sync with the external database. When a Document opens it collects all the existing annotation family instances and stores them in a global list using their ElementIds. I have setup the DocumentChanged event set to track whether new instances of the annotation family have been created or previous instances have been deleted (by comparing the deleted ids). I would prefer to track the elements via UniqueId but DocumentChanged only lists deleted elements via ElementId. If a change is flagged, I then subscribe to the IdlingEvent and store the Document HashCode and ElementId of the affected host View in global variables.

When IdlingEvent triggers I first verify the global variable of the Document is still open by comparing the stored HashCode with the open Document HashCodes. If I find a match, I then verify the ElementId is still a View element in the document. If both are ok I then collect all the family annotation instances in the view, verify their parameters match what the external database is showing, sync the elements with the external database, and redraw the schedule on the sheet.

Hopefully this is making some sense. To recap, I need to do the following:

1. Verify the parameters in the AnnotationSymbol specified match the properties set in the external database.
2. Verify additions and deletions of the AnnotationSymbol instances are tracked "live" with the external database.
3. Keep the report and schedule on each view current with number of AnnotationSymbols on the view.

I realize this cannot always maintain a truly fool-proof connection with the external database so I also have an External Command that basically does the same thing that the Idling Event does.

I welcome any suggestions to deal with unique problem.

Answer: I still think that the DMU+DCE combinations is a superior solution to using IE+DCE.

Here is how it should work:

• On DMU, look for added and deleted glass annotations. If there are any, sync with the database and update the schedule in the model as needed
• On DCE, only do something if the operation was Undone, or Redone, or GroupRolledBack. Ignore the other operations. When there is a change in added or deleted glass annotations, update the database but do not change the model (because it is being taken care of automatically by Revit)
• There will be no need for having global variables storing the document and/or view

Like I said, the IE+DCE can be used, in theory, but:

• It is less natural an implementation and more complex too
• There is a lot more the application needs to look for
  • There is no guarantee Idling would be raised after DCE and before a document is saved or closed
  • Therefore subscribing to DocumentSaving, DocumentSavingAs and DocumentClosing
  • Event with those additional events it is not guaranteed a document would be always in sync with the external database when it is saved
• It is much less efficient, especially with a lot of undos and redos, because the external application must do what Revit would have otherwise done automatically and far more efficiently.
• The Idling event, though it looks like a viable solution in many cases, it is a good approach in a few specific scenarios, mostly involving modeless dialogues and work threads. Beside these very specific scenarios, there is always a better way – more efficient, more practical, and easier too.

Again, the rule for this and similar patterns is rather simple:

1. If you need to react to a changes in a model by making other changes to it, use DMU
2. If you need to react to changes in a model by making changes in an external database, use DCE
3. If you need both, you use both DMU and DCE, but you still follow the above tactic

Response: Although I really like the intent of DMU, I still see some major limitations of it in 2012:

1. Inability to create new transactions inside the DMU
2. Inability to create group elements inside the DMU
3. DMU does not get called for rolled-backed or redone transactions (check me on this one)

Because of these limitations, I ended up using the (DCE+DMU)+IE for optimum performance. The DCE and DMU events tracked changes to the model and called upon the IE when required. The DCE tracks changes to AnnotationSymbols and DMU tracks changes to a specific View parameter. The IE provided me the ultimate freedom to create transactions as I wish. I understand there is a risk for things to change after DCE and before the IE. Although the window for change is small, each programmer should decide how they want to deal with that risk.

Answer: I beg to disagree about the notion of DMU 'limitations'. Those listed above are not limitations, since DMU was designed exactly that way. No one is supposed to need a transaction and/or group inside a DMU, unless DMU is misused. Also, DMU does not need to be called at the event of Undo and Redo – if someone appears to need it, that someone does not use DMU correctly or is not clear on what the purpose of DMU is. I can only repeat what I wrote previously:

DMU – is for changes to the model in effect of other changes in the same model. All the changes will be and should be inside one transaction, because they are – obviously – related, which also mean that if one change is undone, the other (initial) changes need to be undone too. In another way – DMU is an extension of Revits regeneration, the last regeneration that always happens at the end of every transaction.

DCE – is for changes to external data reacting to modifications to the model. No transactions are needed here and the model is not supposed to be touched.

I agree that IE gives the ultimate freedom; I disagree that the freedom should be utilized in this particular case, but I also made it clear that it was 'I' who would not use IE if 'I' was the one solving this particular problem. I do not want to tell users what they have to do – they should have the freedom to do whatever they desire. I just provide an explanation for why one solution is more favourable than the other. IE has very limited use when it is efficient and useful – beyond that recommended use it becomes not so efficient and troublesome to maintain.

Extraneous Undo Entries

There is one more thing I do not 'like' about the Idling event, or – more precisely – about transactions created in the Idling event handler, when it is used outside of the recommended patterns (modeless dialogue, work threading, etc.). The transaction may come as a surprise to the unaware end user. The user pauses for a while, maybe leave her desk to get a cup of coffee, and when she returns there may be three new transactions on the undo stack. If I was her, I would most likely not like it. In contrast to that, changes made during DMU are expected – they are part of the action that is currently going on. There may not be transactions during DCE, so no surprises there. And if Idling is used to implement commands in a modeless dialogue, the user would not be surprised by an eventual transaction. In API programming it is not always about what way appears to be helpful to the programmer – it is about the end user. Revit addition should be made to best follow the natural flow of things. At least in my opinion.

Response: Yes I agree IE isn't the prettiest solution to most problems. The "mystery" transaction(s) created by the IE do affect the natural flow of the program. I tried my utmost to keep the IE duration as minimal as possible. I also took particular care to ensure I only created one transaction during the IE update. I'm sure the user would be shocked to see multiple phantom transactions created repeatedly.

My earlier use of the word "limitation" with regards to the DMU was probably misguided. For my particular work-flow, in my opinion, the DMU was simply not versatile enough. I needed to create groups on the fly and this is simply not permitted inside the DMU. I forgot to mention I also tried creating new families (in place of the groups) and load them into the project during the DMU but again, this is not permitted. I think this was the only thing that prevented me from using DMU+DCE without the IE.

Answer: Regarding your statement "only created one transaction during the IE update. I'm sure the user would be shocked to see multiple phantom transactions created repeatedly":

I did not even mean that one application would create multiple transactions in one single Idling event. But if more add-ins were written this way, and if the users happened to have, say 10 of them, there may be 10 or more transactions during one Idling, and a lot more in one coffee break (with Revit left unattended). It is because one change could lead to another (in a different add-in), which may lead back and trigger yet another change or changes in the original add-in, and so on. Everything would be tied together so tightly so it could be quite a mystery to the end user to figure the course of changes. And when the user decided to undo just some of the involuntary transactions and if the synchronization between Idling and DCE in the individual add-ins is not written perfectly and robust, the hell may as well open for the user (and her document) ;-) I am kidding, of course, but the fact remains that such situations happen in the real world and it is very hard to maintain. That is why I always recommend keeping transaction short and the changes atomic. Splitting one change into one-transaction-now and another transaction-later is almost always opening the door for troubles.

Again, this is my recommendation only.

Response: I think we all prefer Revit not open a gateway to hell but it's nice to know it has that much power! :)