Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V1next history log #578

Open
wants to merge 44 commits into
base: v1-next
Choose a base branch
from
Open

V1next history log #578

wants to merge 44 commits into from

Conversation

isc-shuliu
Copy link
Collaborator

@isc-shuliu isc-shuliu commented Oct 3, 2024

Implement #366
TODO List

  • Figure out what "committed" is
  • Work out the "history" command
  • Add public API for querying history
  • Record recursively loaded/installed modules
  • Properly implement "commit"
  • Allow more matching patterns when filtering histories
  • Display log entries in a prettier manner
  • Command to see the details of a history log by ID
  • Command to show supported columns for filtering
  • Allow specifying direction for sorting and limit max number of rows

@isc-tleavitt
Copy link
Contributor

isc-tleavitt commented Oct 4, 2024

Re: "committed" - this would track, in non-dev mode, if installation was attempted but failed in some way - that is, it would be set to 0 in this case and 1 if installation succeeded. The reason for the failure (probably a %Status) would be worth recording as well for a failed installation.

@isc-kiyer
Copy link
Collaborator

Re: "committed" - this would track, in non-dev mode, if installation was attempted but failed in some way - that is, it would be set to 0 in this case and 1 if installation succeeded. The reason for the failure (probably a %Status) would be worth recording as well for a failed installation.

This comes into the overall lifecycle rework where we flag the phase in which a module reached during install and track the error that occurred as well. For modules in dev mode, can permit resuming install from the failed phase onwards and for non-dev mode, always start from scratch for the module.

@isc-shuliu
Copy link
Collaborator Author

isc-shuliu commented Oct 8, 2024

Not sure why 2c240de doesn't work.

I have a local registry with modA and modB, where modB depends on modA. When installing modB, IPM will call %IPM.Utils.Module:LoadDependencies(), which uses %SYSTEM.WorkMgr to spawn a new process that runs ..LoadModuleReference().

The change in commit 2c240de is simple. We create a %Persistent history log object, save it in database, and pass the ID to the work unit. In the work unit, we save the success/failure and TimeEnd.

However, when calling %Save() from the spawned process, it raises the error Failed to acquire exclusive lock on instance of '%IPM.General.History. Why is there any lock involved?

@isc-shuliu
Copy link
Collaborator Author

isc-shuliu commented Oct 8, 2024

It appears the lock is created because we're in a transaction. (Tried stepping through the code while monitoring management portal, lock disappears after manually running TCOMMIT)

@isc-shuliu
Copy link
Collaborator Author

Here's how it currently looks like
image

@isc-shuliu isc-shuliu marked this pull request as ready for review October 10, 2024 17:36
Copy link
Contributor

@isc-tleavitt isc-tleavitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few really minor comments, but I think this is great overall.

src/cls/IPM/General/History.cls Show resolved Hide resolved
src/cls/IPM/General/History.cls Outdated Show resolved Hide resolved
src/cls/IPM/General/History.cls Show resolved Hide resolved
@isc-shuliu
Copy link
Collaborator Author

@isc-tleavitt
Based on your feedback, I changed the time diff display to always show seconds and exposed API queries instead of API methods.

Currently, the table data is shared among all namespaces where %IPM is mapped. Do we want to separate them?

@isc-tleavitt
Copy link
Contributor

I think this is good now, just need @isc-kiyer's review

Copy link
Collaborator

@isc-kiyer isc-kiyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@isc-shuliu This is a great and exhaustive feature! My comments and some questions are up

src/cls/IPM/General/History.cls Show resolved Hide resolved
src/cls/IPM/Main.cls Outdated Show resolved Hide resolved
src/cls/IPM/Main.cls Outdated Show resolved Hide resolved
<parameter name="argument" required="false" description="Argument for `details` command" />
<example description="Show history of all packages in the current namespace in descending order">history find </example>
<example description="Show history of all packages in the current namespace where command starts with &quot;load&quot;">history find -DCommandString="load*"</example>
<example description="Show history of all packages in the current namespace where start time is later than 2000-01-01 00:00:00">history find -DTimeStart=">2000-01-01 00:00:00"</example>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than using -D modifiers, may be nice to be able to add sub-modifiers and enum for parameters or modifiers. Then for history, can have enum for the action parameter and sub-modifiers tied to different parameter enums. Is a bit extensive and invasive but I think it will be helpful for many other commands.

Happy to talk through design of this

src/cls/IPM/Storage/QualifiedModuleInfo.cls Outdated Show resolved Hide resolved
Write !, "- ", col
}
} ElseIf action = "delete" {
Do ##class(%IPM.General.History).%KillExtent()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For delete, should be able to filter what deletes. Such as by time or at a minimum by specific namespace. Should require passing another parameter/modifier to indicate what the scope of delete is

/// Get the history of all installations, uninstalls, and loads in given namespace
/// The filter argument is a multidimensional array with structure
/// filter(columnName) = value
/// Where value can optionally start with >, >=, <, <=, =, <> or contain * to indicate a wildcard
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than value starting with such things, perhaps make value a $ListBuild? The possible issue with it starting with that is that the property's value may actually be using one of these symbols.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea, my concern is that It can be cumbersome for users to construct a command string that contains $ListBuild()s. They would need to do something like

zpm "history find -DTimeStart=" _ $lb(">", "2000-01-01 00:00:00")

And I can't think of a way for it to work in interactive mode:

USER>zpm

=============================================================================
|| Welcome to the Package Manager Shell (ZPM). Version:                    ||
|| Enter q/quit to exit the shell. Enter ?/help to view available commands ||
|| Current registry https://pm.community.intersystems.com                  ||
=============================================================================
zpm:USER>history find -DTimeStart=???

src/cls/IPM/General/History.cls Outdated Show resolved Hide resolved
src/cls/IPM/General/History.cls Outdated Show resolved Hide resolved
src/cls/IPM/Main.cls Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants