User:Joeytwiddle

My userscripts are here:

• http://userscripts.org/users/89794/scripts

and here:

• http://hwi.ath.cx/cgi-bin/viewcvs.cgi/other/gm_scripts/

There is some discussion about extra standards for userscripts and their API below.

Greasemonkey has really done a great job of improving the web experience for many users. But I wonder if a little bit of polish could finally round off the userscript experience.

Maybe this isn't the right place to ask, but what are the cool kids using these days? I tried Stylish but I found it more klunky to use than Greasemonkey.

GrimeApe - an HTTP Proxy acting like Greasemonkey

During 2009 I wrote an emulation of Greasemonkey in a proxy, so that I could use GM userscripts in Konqueror ... I called it GrimeApe. It has a homepage on http://hwi.ath.cx/ or you can test it immediately simply by using the proxy hwi.ath.cx:7152 . I am curious whether we can get a reasonable level of security using this injection method (I believe we can hide from other JS code through closure, and secure connections using a hidden session key). This proxy should theoretically be able to work with any browser, but so far only Konqueror and Firefox have been confirmed working, and IE6 confirmed broken.

Unfortunately I don't have a lot of time to check this out, but there is also a Konqueror 3.5 plugin that supports this, but as cross platform userscripting doesn't appear to the be focus of the GM wiki, I haven't added it yet. You might want to add it to Cross-browser userscripting and explain it a little more in detail. Make note the TOC is currently alphabetized. Marti 02:07, 8 August 2009 (EDT)

Bookmarklets vs Userscripts

I still use Bookmarklets. They are great when browsing unknown sites and I want to do just a little tweaking to the page. Some of the ones I use are here: http://hwi.ath.cx/joeys_bookmarklets.html

Greasemonkey gave userscripts all the powers that Bookmarklets were lacking, persistent state, auto-execute and cross-domain XHR, hooray! But sometimes I feel that GM has an extra layer of hoops to jump through. Bookmarklets are quick and flexible. You don't have to specify which sites a Bookmarklet should load on. You can be browsing on any website and invoke the Bookmarklet you need when you need it. (The current GM policy is that users must define which URLs a userscript should run on, before loading the page. Inconvenient!)

I sometimes dream about a merger of the two technologies, and in my head it appears like this:

• On every page I surf, Greasemonkey loads up my "Userscript bar". Initially this is just a little icon (maybe a star or a monkey face) which sits in the corner of the page. When I click it or mouseover, the bar unfolds.
• Sitting in my userscript bar are all my favourite userscripts. They each have an icon. The ones which won't work on this page are greyed out or hidden. But basically I can click on the icon of a userscript when I want to use it.
• I would imagine these "scriptlets" would meet some standard which would allow them to:
  • Maybe run some things at startup:
    • Maybe the user previously re-arranged this page using this scriptlet, and they want the changes applied on all visits.
    • The scriptlet might show meta-info about a page. It could start loading this in the background, and update its icon to present the results summary, or the full summary when clicked.
    • The scriptlet can take a look at what is on the page, and can disable itself if it cannot be of use here.
  • Have a default action when invoked by user click, or a standardized menu drop-down if the scriptlet has multiple features.
  • Provide an Icon.
  • Still have the ability to respond to @include and @exclude metas, but these will be used slightly less often.
  • Provide an unload hook as described elsewhere. If we have finished using a scriptlet, or it's simply not doing the right thing, we invoke this to unload the script and undo its changes.

What would this achieve?

• I suggested a richer UI. We can interact with all our Userscripts in a fast way. They can present us information through their icon and tooltip. We can just occasionally invoke a rare script that is only useful in odd moments. If it works, we could click [x] Always run on this page or [x] Always run on this site.
• I am suggesting these userscripts become more like mini-apps. Web helper tools, but cheaper than installing a Firefox extension. If you want, think of it as "My Mashup Toolbar".
• Have you other Linux users noticed how website GUIs running inside the browser window are actually faster than Firefox's own chrome/xul GUI with native Gnome/GTK?! Crazy but true.

My dream is still forming. Maybe you have a better dream...

I did once stumble across a userscript which was attempting to provide a common GUI library which other userscripts could employ to provide the user with a configs/settings editor.

Watching AJAX pages

AJAX is becoming more and more widespread. This means fewer total page loads, and more smaller page updates via XMLHttpRequest or JSON or script element injection, which is making for a richer and lighter web.

The difference it makes to us is that userscripts are only run at the loading of a new page. Usually that is the perfect time to search for certain things on the page which we want to change ("targets"). But now AJAX is adding things to the page at unpredictable times *after* the page loads.

Some of my scripts are failing to operate on such newly loaded elements. I should like to find the best method to attack this problem. (This problem doesn't only apply to AJAX, but also for example to other userscripts which add elements to the page *after* our userscript has added its event listeners.)

1. Firefox allows us to attach "DOMNodeInserted" (or "DOMNodeUpdated") events when elements in the DOM have changed. Listening for these would be one approach. But I would really like something cross-browser compatible (or fudgeable, e.g. via Base2).
2. Another approach is not to add event listeners to individual elements during an initial page scan (or later updates). Instead add one capturing listener at the top of the page which will check when it is fired whether it is on a relevant object, and most of the time remain dormant. This should work fine on elements added to the page any time after page load.
   • Pros: fewer events on the page means a simpler event model - less memory.
   • Cons: all events setup like this may get fired on every element the mouse moves over - more CPU! We must make our decision lightning fast or we will slow down the browser.

OK so going for the second approach, I added a capture=true event listener to document.body which listens for events on A elements. But I ran into a problem, which was that if I moused over an EM or SPAN *inside* an A, then the event would not get fired on the A. Anyway I worked round it with a bit of a hack, which I'll post here for you. Basically it checks up the tree for a parent which is an A, and if one is found, it passes a faked event onto that element. The code went a bit like this:

function addGlobalConditionalEventListener(evType,handlerFn,whereToFireFn) {
	document.body.addEventListener(evType,function(evt){
			var finalTarget = whereToFireFn(evt);
			if (finalTarget) {
				var fakeEvt = ({});
				// Let's set the properties I know I need (for (prop in evt) does not work for this!)
				fakeEvt.target = finalTarget;
				fakeEvt.currentTarget = evt.currentTarget;
				fakeEvt.clientX = evt.clientX;
				fakeEvt.clientY = evt.clientY;
				return handlerFn(fakeEvt);
			}
	},true);
};

var linksOnly = function(evt) {
	try {
		var node = evt.target;
		while (node) {
			if (node.tagName == "A")
				return node;
			node = node.parentNode;
		}
	} catch (e) {
		node = null;
	}
	return node;
};

addGlobalConditionalEventListener("mouseover",createTooltip,linksOnly);

This could do with some refactoring, but may be a useful code snippet for GM. I fear however when moving from an A to its child SPAN, we'll probably fire a mouseout then a mouseover, both on the same A! Not quite what we were after, but close enough for some tasks.

Suggestion: Immediate loading of an enabled script

When we bring up the monkey's menu to turn on a script which was un-checked (disabled), we then have to reload the page for the script to be executed. Why?! Since userscripts are not run until after the page has loaded, couldn't GM run the activated script immediately, without the need for a page refresh?

(You don't necessarily need a full Reload. Navigating to the current page via a link or by URLbar->Enter should be enough. However I tend to naturally hit Ctrl+R or the Reload icon, although in FF this causes a full reload which is significantly slower.)

Scripts which would benefit from this are those which we use more like a Bookmarklet. Some scripts can work over a large range of websites, but we usually prefer them disabled. Occasionally we want to use a script for one page, or for a short while during a browsing session. Sometimes with such scripts, the solution is for them to add a link or button to the page, and then lie dormant, only activating when the link or button is clicked.

Solved by @require: Library Userscripts and Dependencies

Suggestion Withdrawn: @depends metadata

It could be useful to have an extra meta tag:

// @depends    jims_js_library>=2.0
// @depends    temporary_iframes, floating_widgets>=1.2.5
//   or
// @loadlib    <script_name>

The @depends meta would cause Greasemonkey to load the specified script from the user's gm_scripts/ subfolder before running the userscript which contained the meta.

This would allow us to define a library script of common functions which we can re-use in any of our userscripts, simply by adding the meta. These library scripts are always loaded from a trusted source, they are local userscripts which do nothing when run alone.

The @version of the target library scripts could be checked against the conditions defined by >= in @depends. The user or the update system could be informed if the requirements are not met.

Sounds like a good start to a feature request in case the sandbox isn't ever expanded... you may wish to enter an enhancement ticket at http://greasemonkey.devjavu.com/newticket with this instead of the wiki talk pages. It will get the proper attention there. Marti 04:46, 7 June 2009 (EDT)

Thanks Marti, I see that is where the development discussion is. The greasemonkey-dev mailing list is to be overwhelmed with user-like questions!

Aaron also contacted me last week while I was out at conference and the new ticket home is at http://github.com/greasemonkey/ Things are still not fully set up there and hopefully it will get the attention that your ideas and presentation deserve. Marti 02:02, 8 August 2009 (EDT)

I finally noticed the @require meta! This appears to do exactly what I was after. Documentation Discussion. The suggestion made above only differs in that scripts are more trusted because they are local, and the meta syntax could help track versioning and updates. Joeytwiddle 12:50, 21 August 2009 (EDT)

Deprecated by @require: Current techniques for loading library userscripts

It is already possible to use userscripts as libraries by a couple of techniques:

Pre-embed with GM

Have one userscript load early, and let it embed some new JS functions into the unsafeWindow. Later userscripts may check the window to see if the functions it wants to use are available.

Or create an Object full of useful functions, and embed it in the DOM. Any userscripts which execute later could reference the object to make use of those functions.

We would somehow need to ensure the library scripts run first, and that they run on all pages where one of their dependent scripts will run.

Unfortunately this raises security issues since untrusted scripts can now access our globally visible object or functions. Maybe we can force our library to drop GM internals from its scope, for example by placing a fn.toString() clone into the page rather than a reference to the actual function.

Load-on-demand via http

When we want to load a library and it is not already in the page. Add a new script element to the page, with src attribute pointing to a publically visible http URL where the library userscript is located. Whether the functions from the loaded script become immediately visible depends. It seems to work fine when done with Bookmarklets in Firefox3.5, but some other browsers may need to wait (via a setTimeout or scriptElement.onload) *before* the loaded functions will become visible to running code.

There may be other techniques...

Given that these techniques exist, does that mean we don't need @depends?

You will probably need to have some more chats with Johan (ecmanaut) and if Aaron continues to make himself more available, him as well. Anthony may also pop in every once in a while too. As I stated later in these replies, GM is a bit behind the times but still works... It is just harder to implement more advanced features from a heavily moderated and "silent" project. Best bet is to try the community that makes GM possible at userscripts.org. You can also read some of the communities responses at this topic Marti 02:45, 8 August 2009 (EDT)

Suggestion: GM_clearStyle()

It is common that you want your userscript to add some floating menus or windows on top of the page. Unfortunately these will often inherit CSS rules from the page, and your added elements can end up with strange colours and alignment. It would be handy to break out of page style settings and create elements using a default empty stylesheet. How might this be possible? Can we literally move to a fresh CSS namespace? Or do we need something like this?

GM_clearStyle(node_element,bool_clearChildren);

Part of my objectives with my involvement in GM is to educate (and be educated myself too) the other dev's into better implementations of existing methods. As you may have observed I don't always hit the right chord with everyone... however my dedication is still here. From my Mozilla experience it is possible to move to a separate namespace on the CSS however GM still needs some serious refactoring before this can happen. This unfortunately currently limits GM in several areas and is why several authors have forked GM.

I am also taking your "clearStyle" a few different ways... the main being overriding CSS inheritance... please correct me if I'm misunderstanding this.Marti 02:33, 8 August 2009 (EDT)

You are correct I am wanting to override any inheritance.

Hmmm... well I'm not currently aware of any way to do that. There is a reference to resetting to the defaults via initial. Would this be what you are looking for? Marti 06:30, 14 August 2009 (EDT)

That certainly looks like a way to get the default value. The problem is getting a list of all the CSS property names. There is one tehnique here, and YUI offers a similar file. I am getting a list of 160 names when I do getComputedElement(anyNode,). Joeytwiddle 21:56, 2 September 2009 (EDT)

Another technique is to put your elements into an iframe: ​​HTML_injection_tips 82.45.8.208 19:09, 13 January 2011 (UTC)

Suggestion: Track homepage of installed scripts (for docs, updates)

I often want to visit the homepage of a script I have installed, either for documentation, or for updates, but the author didn't include it. Not all userscripts are installed from userscripts.org, and even those that are can be difficult to find again. I would like Greasemonkey to make a record of the update URL and the homepage of a script when I install it. We could insert the meta tags if they were not included by the author, or inaccurate. There are update scripts which try to track scripts, but shouldn't it be supported?

Again I can't speak for everyone, but this would lead to some privacy concerns here. A few dev's have already stated it would be nice to drop the config.xml altogether in favor of some more favorable method... however no one has come up with a better idea as of yet. Oliver is working on a fork that may eventually address some of this but as of yet I have yet to see the proposed logic. Marti 02:28, 8 August 2009 (EDT)

Suggestion: What is Namespace good for? Drop it from the log!

Finally a thought on logging and @namespace. The only standard I have really seen @namespace used for is to provide the homepage of the script or the author in @namespace. Should this be officially declared as the intention of @namespace? And since these URLs are often long, does it really make sense to make them part of the headers of log messages? Wouldn't just the name of the script be more appropriate when reading the log?

See @namespace for different implementations of this. Part of metadata is for examination purposes and tracking both internally to GM and externally and everywhere in between. It is unlikely that it will be removed due to legal matters regarding metadata, but GM can surprise me every once in a while by ideas that aren't quite up to par. Marti 02:22, 8 August 2009 (EDT)

I can see it might be useful in the log if users do have the same script installed under different namespaces. I don't usually have that. To reduce the width used in the log, I will just set the namespace to something short, and leave @homepage to provide a real link back to the source.

Suggestion: Userscripts should have a standard unload mechanism

It would mean when we disable a userscript in the monkey menu, the script could automatically undo its changes to the page, rather than the user having to reload the page.

The mechanism for doing it could work a little like GM_registerMenuCommand: GM_offerUnload(myUndoFunction);

Contra Development

I'm actually quite happy that Greasemonkey has kept its API small, this can accommodate wider compatibility. I think any additions to the API should be discussed over time by developers and users. I was wondering whether the developers were intending to extend Greasemonkey in the future, or have decided to keep it small and tidy, and leave the addition of features (aka bloat) to other projects.

I can't speak for anyone else but I can report an observation and a link here. Ideas and suggestions are welcome by a certain portion of the community. I've noticed you've caught Johans eye and that is a plus for your contributions. I appreciate you working within the community and being objective enough to present your ideas in a professional manner. I can report also that Greasemonkey doesn't change a whole lot over time, but that is always subject to changing. The dev group (mailing list) is usually quite vacant but not all the time. I can tell you from first hand experience, they don't like "noise" as they put it. This constitutes asking too many questions when the primary authors are in a bad mood (no offense intended just an observation)... so use your best judgement and don't be afraid to ask other people such as the community that makes GM possible at http://userscripts.org. Marti 02:23, 8 August 2009 (EDT)

Slow scripts and worker threads

A few of the scripts I use lock up the browser for a while on large pages. With some of these I have delayed their execution for a few seconds with a setTimeout, so I get to interact with the page before the script does its stuff.

Worker threads have been available in Google Gears for some time. But in Firefox 3.5 they are now available to all users. How can userscripts makes use of worker threads?

It might be difficult to force all GM scripts to run in a worker thread, since they run in a closed environment, so interactions with the DOM would need to be proxied. But scripters who write heavy scripts may want to make use of worker threads.

Undocumented edits

Hey JoeyTwiddle... You made a few changes to the Cross-Browser scripting article. Please ALWAYS include the summary of what you are doing. I don't patrol that article a whole lot, but it would be nice to let everyone know what you are doing rather than having to reverse engineer what you did. There is a setting in Preferences to have the wiki remind you to set it. Thanks :) Marti 13:44, 3 September 2009 (EDT)

• I have the option "Prompt me when entering a blank edit summary" set but it didn't stop me! Joeytwiddle 02:15, 5 September 2009 (EDT)
  • It's working today, and stopped me from causing trouble.  :) Perhaps I was suffering from BetterCache, which rocked aside from the problems.

GM_set/getValue versus localStorage

Has anyone compared the efficiency of these two storage methods? I try to keep my userscripts' persistent data to a minimum, for fear of overloading my hard-working browser. But sometimes I want to store a lot of data and wonder how scalable the storage methods are.

Update: Empyrical observations suggest that localStorage in Chrome is more efficient than GM_get/setValue in Greasemonkey. So I gave my caches appropriate sizes depending on the browser.

Cleanup in Chrome?

Also I'm curious how Google Chrome deals with cleanup of script data, considering it doesn't have GM_deleteValue! I fear my Chrome userscript data cache may be growing linearly with time! (I usually implement a cache which removes the record with the lowest(value / age) score when capacity is exceeded.)

Cross site GM_xmlhttpRequest for Chrome

Since Chrome also lacks cross-site GM_xmlHttpRequest, I'm thinking of implementing it through a JSONP proxy. I have created a prototype using Node: http://hwi.ath.cx/code/other/javascript/xhr_via_json/