Synapse stuff - Revision history

Apheori: Bai.

2013-05-03T20:42:48Z

Bai.

← Older revision		Revision as of 20:42, 3 May 2013
Line 1:		Line 1:
	This document outlines ~~the~~ tasks completed or attempted while working on ''Synapse'' - effectively akin to a research journal but as part of a production-intended project instead of a more specific study.		This document outlines tasks completed or attempted while working on ''Synapse'' - effectively akin to a research journal but as part of a production-intended project instead of a more specific study.

	It was originally in a [https://docs.google.com/document/d/1HQUIQRbYrcEkHHdlrMjw0MLuEbF_aQz4IuN6_1zycxQ/edit google doc], but ~~got~~ moved here ~~because setting up file uploads with a half-configured mediawiki install seemed easier than figuring out a pile of unlabelled buttons on said google doc~~.		I came on toward the beginning of the project, when it was largely only an idea - the general audience, proposed functionality, and possible directions were reasonably established, but measurable direction of the project was uncertain. Initial research helped pin down what functionality was plausible while the others interviewed potential users, and from there we moved onto the MVP as would be presented more broadly in order to assess user pull. Amidst more research, prototype designs were proposed, and Houman implemented the first app iteration. From there, more feasibility research was done for the chosen platform, google drive, and also for audio handling in general, though less was found regarding that than would have been hoped. A final round of usability and performance testing served to wrap up my involvement with the project, and while the results brought up issues that merit further looking into, they also indicate there is indeed reasonable hope for Synapse as a viable product.

			The journal was originally in a [https://docs.google.com/document/d/1HQUIQRbYrcEkHHdlrMjw0MLuEbF_aQz4IuN6_1zycxQ/edit google doc], but was subsequently moved here for accessibility purposes.

	== Task 1 - usable existing products ==		== Task 1 - usable existing products ==

Apheori: /* Task 6 - usability and performance testing */

2013-05-03T06:06:58Z

Task 6 - usability and performance testing

← Older revision		Revision as of 06:06, 3 May 2013
Line 158:		Line 158:
	;There's no good way to record gestures, examples demonstrated with an object (like holding up a pen and showing how it can fall in different situations), etc, which may be relevant. Pictures better than nothing, maybe, but just not the same.		;There's no good way to record gestures, examples demonstrated with an object (like holding up a pen and showing how it can fall in different situations), etc, which may be relevant. Pictures better than nothing, maybe, but just not the same.

	;~~lass~~ structures can present issues.		;Class structures can present issues.
	* Makes sense for lectures.		* Makes sense for lectures.
	* Can make sense for presentations, but need a way to mark who is presenting, when presenters transition, etc.		* Can make sense for presentations, but need a way to mark who is presenting, when presenters transition, etc.

Apheori: /* Task 6 - usability and performance testing */

2013-05-03T06:06:27Z

Task 6 - usability and performance testing

← Older revision		Revision as of 06:06, 3 May 2013
Line 145:		Line 145:

	== Task 6 - usability and performance testing ==		== Task 6 - usability and performance testing ==
	Record ~~four~~ lectures - get volunteers to test the app while also recording it ourselves.		Record three lectures
	* Multiple recordings increases pool of audio we have to work with and test future functionality.		* Planned to get volunteers to test the app while also recording it ourselves. This worked for one (performance modelling), another did not (startups class; Arjun asked the class to record from laptops, but nobody actually did), and third was just us recording from phones (chaotic dynamics), but in a room with the whole technical setup specifically for recording, so we could compare results.
	* User testers can find problems with usability and also give insight into usage patterns and different use cases.		* Rationale: Multiple recordings increases pool of audio we have to work with and test future functionality. User testers can find problems with usability and also give insight into usage patterns and different use cases.

	=== Things that come up ===		=== Things that come up ===
	Taking pictures is kind of awkward.		;Taking pictures is kind of awkward.
	* Picking up idling phone in general and holding it up is strange behaviour in a class.		* ''Picking up idling phone in general and holding it up is strange behaviour in a class - feels strange to do, and potentially distracting to the lecturer as well, since a student is holding up a phone randomly.
	* Phone likes to go to sleep; unlocking it takes time and can prevent actually getting the picture.		* ''App itself lagged while holding it up to take the picture, which made it more awkward (probably lagged due to changing orientation)
			* ''Phone likes to go to sleep; unlocking it takes time and can prevent actually getting the picture.
	* Folks tended not to take pictures.		* Folks tended not to take pictures.

	~~Class~~ structures can present issues.		;There's no good way to record gestures, examples demonstrated with an object (like holding up a pen and showing how it can fall in different situations), etc, which may be relevant. Pictures better than nothing, maybe, but just not the same.

			;lass structures can present issues.
	* Makes sense for lectures.		* Makes sense for lectures.
	* Can make sense for presentations, but need a way to mark who is presenting, when presenters transition, etc.		* Can make sense for presentations, but need a way to mark who is presenting, when presenters transition, etc.
	** (There was a comment from Zach about how recording things is also useful for the presenters to improve their presentation skills)		** (There was a comment from Zach about how recording things is also useful for the presenters to improve their presentation skills)

	What if speakers are on all sides of the room? Some quieter than others, talking in different directions... (~~haven~~'t ~~had~~ a ~~chance~~ to ~~listen~~ to ~~results yet so~~ this ~~is supposition~~, ~~though)~~		What if speakers are on all sides of the room, or just students asking questions? Some quieter than others, talking in different directions... (Something to pay particular attention to when examining test results? Definitely came up in the startups and chaos classes.)

			=== Chaos dynamics ===
			This lecture takes place in a fancy room specifically set up for recording. Said setup ran into trouble while we were also there recording from the app, where the recording system failed. Speakers cut out and it was uncertain what all was working and not - thought perhaps the recording was still working and that just the speakers had gone out, but it turned out to be both and five minutes of lecture were lost as a result. The folks running the thing were somewhat frantic as well, with apparently no clear feedback on either end as to what all was working/not working.

			So feedback should be a concern - something in the app itself to tell people it's working (maybe just a displayed wave of the recording, or show the levels, or whatever), and something they would notice when it quits working. This is a valid reason to sacrifice simplicity of the interface, because above all else users want the thing to work, and if it isn't working, they want to know. They don't want to lose stuff.

			Lecture itself is done digitally. Looked like a presentation on a tablet, professor drawing curves on and adding notes to slides.
			* Add screencapping software to supplement the app for the presenter machine, which tracks timestamps for later syncing with audio?

			=== Note on audio/video syncing ===
			Nevermind the fancy audio analysis needed to find splice points, should probably '''line up the separate audio recordings based on timestamps''' in general, and this would work for video/slides too. These apps should all be on internet-enabled devices, so they should be able to sync timestamps with a central server while recording and include those in the upload.

Apheori: Whee!

2013-04-24T21:20:42Z

Whee!

← Older revision		Revision as of 21:20, 24 April 2013
Line 143:		Line 143:
	* Do Echoprint or AcoustID allow to find the exact overlapping points of two audio files or all they tell is if the fingerprints match up or not?		* Do Echoprint or AcoustID allow to find the exact overlapping points of two audio files or all they tell is if the fingerprints match up or not?
	*: ''From what I could find there really aren't any (effectively advertised) solutions to what we want to do. Echoprint and AcoustID are intended to check if files are the same, but because they do so by matching fingerprints, these fingerprints could also be adapted to our purposes to check for partial overlap and sameness as well, thus providing the overlap points.		*: ''From what I could find there really aren't any (effectively advertised) solutions to what we want to do. Echoprint and AcoustID are intended to check if files are the same, but because they do so by matching fingerprints, these fingerprints could also be adapted to our purposes to check for partial overlap and sameness as well, thus providing the overlap points.

			== Task 6 - usability and performance testing ==
			Record four lectures - get volunteers to test the app while also recording it ourselves.
			* Multiple recordings increases pool of audio we have to work with and test future functionality.
			* User testers can find problems with usability and also give insight into usage patterns and different use cases.

			=== Things that come up ===
			Taking pictures is kind of awkward.
			* Picking up idling phone in general and holding it up is strange behaviour in a class.
			* Phone likes to go to sleep; unlocking it takes time and can prevent actually getting the picture.
			* Folks tended not to take pictures.

			Class structures can present issues.
			* Makes sense for lectures.
			* Can make sense for presentations, but need a way to mark who is presenting, when presenters transition, etc.
			** (There was a comment from Zach about how recording things is also useful for the presenters to improve their presentation skills)

			What if speakers are on all sides of the room? Some quieter than others, talking in different directions... (haven't had a chance to listen to results yet so this is supposition, though)

Apheori: /* More questions */

2013-04-22T18:49:00Z

Apheori: /* Task 5 - feasibility stuff */

2013-04-19T18:20:08Z

Task 5 - feasibility stuff

← Older revision		Revision as of 18:20, 19 April 2013
Line 133:		Line 133:

	A paperly overview of some of the technology can be found here: [http://ucbmgm.googlecode.com/svn-history/r7/trunk/Documentos/Fingerprint-Cano.pdf Fingerprint-Cano.pdf]		A paperly overview of some of the technology can be found here: [http://ucbmgm.googlecode.com/svn-history/r7/trunk/Documentos/Fingerprint-Cano.pdf Fingerprint-Cano.pdf]

			=== More questions ===
			* How can we add a sentence to a Google doc using the api? (to allow students to ask anonymous questions.)
			* How can we replace a word in a Google doc using the api? (why? we'll have templates for new created Google docs. Each template will have some placeholders (lecture title, instructor name) on top of the page which need to be replaced with real data from the corresponding lecture.)
			* Does the Google API allow to turn off the chat feature?
			* Do Echoprint or AcoustID allow to find the exact overlapping points of two audio files or all they tell is if the fingerprints match up or not?

Apheori: /* Task 5 - feasibility stuff */ Good gods, google is redefining horrible UX at every turn.

2013-04-13T18:09:35Z

Task 5 - feasibility stuff: Good gods, google is redefining horrible UX at every turn.

← Older revision		Revision as of 18:09, 13 April 2013
Line 103:		Line 103:

	== Task 5 - feasibility stuff ==		== Task 5 - feasibility stuff ==
	* How to parse a google doc (API results, etc, what would be pulled out)?		* How to parse a google doc (API results, etc, what would be pulled out)?__NOINDEX__
	* Something about how to match up snippets of audio from different sources - algorithms, established methods, possible products, etc.		* Something about how to match up snippets of audio from different sources - algorithms, established methods, possible products, etc.

			=== Parsing google doc ===
			From [https://developers.google.com/drive/manage-downloads#downloading_google_documents developers.google.com]:

			Download the content as described (probably as plain text since we're unlikely to care about images, text format, etc for this).

			To parse, regex would probably be more than sufficient:
			* Find sentences ending with questions
			* Find hashtags
			* Etc


			Need to have a background process on the server or something to update - use the [https://developers.google.com/drive/v2/reference/changes/list changes feed]; check [https://developers.google.com/drive/manage-changes new changes](?) and reparse as needed.

			=== Matching audio clips ===
			The aim is to find overlap between multiple different recordings in order to combine disjoint recordings of the same meeting into a single, higher-quality file.

			We probably want to match the [[wikipedia:acoustic fingerprint\|acoustic fingerprints]] of audio files as the first part of the process. Normally this sort of technology tends to be used for identifying music, but applications developed for that may work for this as well. In essence a fingerprint is a simplification of an audio file that focusses the perceptual characteristics (what a person would hear), and from this a basic comparison allowing for a certain range of variance between file fingerprints should serve our needs.

			An existing open source solution such as [http://echoprint.me/ Echoprint] or [http://acoustid.org/server AcoustID] would probably be ideal if they would work.

			The problem is how to figure out what, if any such option, ''would'' actually work.
			* Recordings from different devices and places will contain considerably more variance and differing noise than reencodings of a song.
			* Depending on how an implementation handles partial recordings, these may or may not even work to find overlap in the recordings.

			Alternately generating our own 'fingerprints' for each snippet and then running them through a sloppy comparison possibly using the same methods as to generate the fingerprints in the first place - are they similar enough? - to match them up might be another, potentially more robust, option.

			A paperly overview of some of the technology can be found here: [http://ucbmgm.googlecode.com/svn-history/r7/trunk/Documentos/Fingerprint-Cano.pdf Fingerprint-Cano.pdf]

Apheori: Stuff

2013-04-10T19:44:44Z

Stuff

← Older revision		Revision as of 19:44, 10 April 2013
Line 33:		Line 33:
	** Specifically created for integrating into other things, uses .Net/js, free for academic/NC use - would probably work fairly well without too much trouble.		** Specifically created for integrating into other things, uses .Net/js, free for academic/NC use - would probably work fairly well without too much trouble.

	== Task 2 - questions ==		== Task 2 - MVP demo ==
			Work on script, record an audio...

			Fiddled with it, and came up with something which fortunately wasn't used.

			== Task 3 - questions ==
	How can we get people signed up for our private beta?<br>		How can we get people signed up for our private beta?<br>
	How can we run A/B testing for our demo?		How can we run A/B testing for our demo?
Line 75:		Line 80:
	If that doesn't work it's hopeless. Or something is wrong, at any rate.		If that doesn't work it's hopeless. Or something is wrong, at any rate.

	== Task 3 - prototypes ==		== Task 4 - prototypes ==
	Mockups/prototypes to give folks a feel for the product...		Mockups/prototypes to give folks a feel for the product...

Line 96:		Line 101:
	File:Notes mockup.png\|Notes page		File:Notes mockup.png\|Notes page
	</gallery>		</gallery>

			== Task 5 - feasibility stuff ==
			* How to parse a google doc (API results, etc, what would be pulled out)?
			* Something about how to match up snippets of audio from different sources - algorithms, established methods, possible products, etc.

Apheori: linky

2013-04-08T18:05:42Z

linky

← Older revision		Revision as of 18:05, 8 April 2013
Line 1:		Line 1:
	This document outlines the tasks completed or attempted while working on ''Synapse'' - effectively akin to a research journal but as part of a production-intended project instead of a more specific study.		This document outlines the tasks completed or attempted while working on ''Synapse'' - effectively akin to a research journal but as part of a production-intended project instead of a more specific study.

	It was originally in a google doc, but got moved here because setting up file uploads with a half-configured mediawiki install seemed easier than figuring out a pile of unlabelled buttons on ~~the~~ google doc.		It was originally in a [https://docs.google.com/document/d/1HQUIQRbYrcEkHHdlrMjw0MLuEbF_aQz4IuN6_1zycxQ/edit google doc], but got moved here because setting up file uploads with a half-configured mediawiki install seemed easier than figuring out a pile of unlabelled buttons on said google doc.

	== Task 1 - usable existing products ==		== Task 1 - usable existing products ==

Apheori at 00:35, 8 April 2013

2013-04-08T00:35:41Z

← Older revision		Revision as of 00:35, 8 April 2013
Line 1:		Line 1:
	This document outlines the tasks completed or attempted while working on ''Synapse'' - effectively akin to a research journal but as part of a production-intended project instead of a more specific study.		This document outlines the tasks completed or attempted while working on ''Synapse'' - effectively akin to a research journal but as part of a production-intended project instead of a more specific study.

	It was originally in a google doc, but got moved because ~~Kim finds~~ google ~~products 'bloody irritating'~~.		It was originally in a google doc, but got moved here because setting up file uploads with a half-configured mediawiki install seemed easier than figuring out a pile of unlabelled buttons on the google doc.

	== Task 1 - usable existing products ==		== Task 1 - usable existing products ==

← Older revision		Revision as of 18:49, 22 April 2013
Line 136:		Line 136:
	=== More questions ===		=== More questions ===
	* How can we add a sentence to a Google doc using the api? (to allow students to ask anonymous questions.)		* How can we add a sentence to a Google doc using the api? (to allow students to ask anonymous questions.)
			*: ''Using a generic user to do the add (such that said user would be what shows up in the history instead of the student by name), the studen sends it the sentence which the generic user then adds using [https://developers.google.com/drive/v2/reference/revisions/patch patch]...
	* How can we replace a word in a Google doc using the api? (why? we'll have templates for new created Google docs. Each template will have some placeholders (lecture title, instructor name) on top of the page which need to be replaced with real data from the corresponding lecture.)		* How can we replace a word in a Google doc using the api? (why? we'll have templates for new created Google docs. Each template will have some placeholders (lecture title, instructor name) on top of the page which need to be replaced with real data from the corresponding lecture.)
			*: ''Probably [https://developers.google.com/drive/v2/reference/revisions/patch patch] for this as well.
	* Does the Google API allow to turn off the chat feature?		* Does the Google API allow to turn off the chat feature?
			*: ''The documentation does not seem to cover the chat feature, at least not that was found. But either it shows up or not when the object is rendered and can be enabled/disabled or not, but the fastest way to check this would probably be to set up the thing itself. Even if chat is there without option when we don't want it, if all else fails we could probably make it go away on our end easily enough.
	* Do Echoprint or AcoustID allow to find the exact overlapping points of two audio files or all they tell is if the fingerprints match up or not?		* Do Echoprint or AcoustID allow to find the exact overlapping points of two audio files or all they tell is if the fingerprints match up or not?
			*: ''From what I could find there really aren't any (effectively advertised) solutions to what we want to do. Echoprint and AcoustID are intended to check if files are the same, but because they do so by matching fingerprints, these fingerprints could also be adapted to our purposes to check for partial overlap and sameness as well, thus providing the overlap points.