To facilitate this collaboration I created a repository already on which we will push our work for this Proof of Concept (Poc): https://github.com/OTA-Insight/default-mobile-browser-emulator

Since upgrading Hero and the like to version alpha-15, I get the following SQL errors at runtime (in production):

Not always, but when I do get it, it breaks the session.

1. toString on a proxy of a proxy was not correctly logging an error originating from an Object
2. UserAgent data was not being properly retrieved

Closes #5

Disables out of process iframe (OOPIF) since those create a weird navigation swap in/out that we don't know how to track properly.

We also fixed lint, which required changes to be up to standard

How to repeat?

const { request: { headers}  } = await hero.reload()
console.log(headers['sec-fetch-site']) 

same-origin is expected, but the browser sends none.

• [x] fail default-browser-emulator for android (mobile) targets:
  • As part of the effort on supporting mobile browsers natively to unblocked, a first good step would be to make sure the default browser emulator does not accept mobile targets (e.g. android chrome).
• [ ] collaborate on a PoC mobile browser emulator plugin:
  • To facilitate this collaboration I created a repository already on which we will push our work for this Proof of Concept (Poc): https://github.com/OTA-Insight/default-mobile-browser-emulator
  • [ ] publish PoC project to NPM
  • [ ] have a (private) project make use of this plugin and report on its usage + iterate while using it
• [ ] copy portions of the default emulator into chrome for android emulator to emulate user agent and activate touch
  • This is where an emulator says it can handle a user agent https://github.com/ulixee/unblocked/blob/d468b4aba38b0f3cd800b1dc241586265ff13a38/plugins/default-browser-emulator/index.ts#L226
• [ ] activate mobile user agent strings in unblocked/real-user-agents
• [ ] study the headers/tls/tcp/etc in double-agent profiles
• [ ] automatically generate double-agent profiles based on our findings
• [ ] add missing evasion techniques
• [ ] refactor common evasion technique functionality

The end goal of this pull request is to use the flag captureBeyondViewport added in chrome 87, and migrate all screenshot logic to this approach. This would solve #13, #21, and in general make the screenshot logic much simpler.

This initial commit is in no way meant to be merged, but is here to show that is now possible using the default/latest chrome in the unblocked repo.

What still needs to be done:

• refactor code
• test if scale still works as expected
• extend tests, and remove saving images to /tmp
• test in which versions of chrome this works (find oldest working version)

These are the images created in tests, using the old and new logic (left and right respectively): Should be able to take a full page screenshot

Should screenshot only the visible page

Should be able to take a viewport screenshot:

Should be able to take a clipped rect screenshot

New test, should be able to take a clipped rect screenshot outside of viewport

Taking a fullscreen screenshot does not always create the intended image. Sometimes sides effect can be noticed, either on the image, or when running with a headfull chrome instance. These effects are cause, by this code:

await this.devtoolsSession.send('Emulation.setDeviceMetricsOverride', {
          ...contentSize,
          deviceScaleFactor: scale,
          mobile: false,
});

Changing the viewport of the device is a clever way to take fullscreen images, but has side effects on website where this change is detected. Some issues that I saw:

• shift of content, probably related to aspected ratio change
• extra context that was opened by hero (querySelector.click...), is closed again
• random weird behaviour in some components

This can partially be linked to #13., and can be solved in newer chrome versions by using captureBeyondViewport: true.

I can take a screenshot as follows:

const mySel = await hero.querySelector('#myId');
const screenshotRectangle =
   // eslint-disable-next-line @typescript-eslint/no-explicit-any
   (await mySel.getBoundingClientRect()) as any;
screenshotRectangle.scale = 1;
const buffer = await hero.takeScreenshot({
   format: 'jpeg',
   rectangle: screenshotRectangle,
   jpegQuality: 100,
   fullPage: true,
});

This gives however the full page, ignoring the given rectangle. If I however leave off the fullPage (which is what I tried first), I get an error which states that the given rectangle is outside the viewport...

https://github.com/unblocked-web/agent/blob/main/core/lib/Page.ts#L770-L773 that's the assert that I get apparently. two observations there:

• for a full page screenshot, you adapt the viewport size to the full page size
• in https://github.com/unblocked-web/agent/blob/main/core/lib/Page.ts#L487 I read that apparently you can capture beyond viewport

So given the second observation I think we can even just drop that assert. But if you do want to support < 87, then I suppose you would want to also support overriding the screen dimensions based on a given rectangle, rather then only doing it for fullPage?

According to @blakebyrnes we might need to resize to fit the element size if it’s that big, see on Discord for his original message: https://discord.com/channels/966293220780806174/966293220780806177/1032432492822667305

This list can serve as a central locator as we create issues to tackle sub-issues

• [ ] browser/dom-worker

  • check full scope of js env in various workers (service, dedicated, shared)
  • [x] collect worker dom environments (NOTE: built collect in browser-dom-environment, but turned off)
  • [ ] analyze dom environments + dom-bridges

• [ ] browser/dom-frame

  • check full scope of js env in various iframes (sandbox, cross-domain, etc)
  • [x] collect in browser-dom-environment (NOTE: turned off and not in browser-profiler-data)
  • [ ] analyze dom environments + dom-bridges

• [ ] browser/tampering

  • look for Javascript proxies, mismatch of dom (Creepjs is great at this)

• [ ] browser/vm

• look for red pills

• [ ] browser/webgl

• [ ] browser/render

  • browser renders dimensions differently per platform/browser

• [ ] browser/voice

  • [x] Collect look for available voices - some are OS specific
  • [ ] needs Analyze plugin

• [ ] http/referrers

• [ ] http/cache

  • measure cache header responses to various server headers

• [ ] http/favicon

  • favicons are requested in headed differently than headless

• [ ] tls/clienthello-ws

  • websockets don't have an alpn for http2 in Chrome
  • [x] this exists in a wss-tls branch, but branch is outdated

• [ ] user/interaction

  • measure interaction with forms/typing/etc for automated patterns. Might need a human alternative

• [ ] user/mouse

  • measure use of mouse to get from one place to another. Some good research papers about this

• [ ] http/headers + http/cookies:

  • Http delete/update (trigger from forms?)
  • Direct loads without referrers
  • Prefetch
  • Signed Exchange
  • Http2 headers/trailers
  • [x] Sec CH-UA headers

• [ ] ip/address:

  • Socket reuse

• [ ] Audio context (new plugin?):

  • https://github.com/cozylife/audio-fingerprint