Week 4 - What I learned

2021/04/27 to 2021/05/03

Overview

This week we continue learning about quantum physics and computation but now we saw son appliances of that. We also got to know how big tech companies do testing and how their culture has changed. We also see how these companies do their real testing within the team. We also learned what is pretotying and chaos engineering. A lot of stuff going on this week.

---

Computing & Science

Seth Lloyd on Programming the Universe

Things to learn or note:

It seems like we are going to here some experiences of Seth working with quantum computers.

• We part from the point that we are in the middle of an information processing revolution based on digital computation electronic circuits.
• We store information in bits, it can be 0 or 1.
• We can find multiple devices to modify the bits, like capacitors or magnets.
• That's how the information processing revolution works you have bits, you can have 0's and 1's and you flip them a lot of times.
• Printers and books are also part of the information processing revolution.
• Spoken word is also important, is what makes us different from other things.
• To communicate we need to have a brain.
• Sex is also part of the information processing revolution. Sex messes with success.
• Sex involves a combination of the genome (information).
• Getting a little bit small, molecules also are part of the information processing revolution.
• Life is about information.
• Each technique of the information processing revolution is built on its previous one.
• The big bang could be the first information processing revolution.
• Now with this we get can get to this idea, universe computes.
• The universe is responsible for other information processing revolutions.
• Quantum computation is using data that the universe is computing. They are making the universe do things like perform quantum computations.
• Quantum computation is once again the result of the universe's natural information processing revolution.
• Quantum mechanics. It's about particles and waves.
• Because of this, we can have 1 and 0 in the same place. (Qubits)
• Wow wow wow... Well, quantum computers have this intrinsic multitasking ability.
• Can do a lot of things at once and they can do it in a bug-free fashion.
• The final answer is just a combination.
• You can build a quantum computer, you could take just a bunch of atoms in a molecule then zap them with microwaves. Each atom in this molecule listens to microwaves responds to microwaves of a slightly different frequency. We can flip the information like this. Just being careful you can make them compute.
• Another way, you can set up a bunch of atoms trap them in an optical cavity, and zap them with lasers put a lot of circuits, and can do computation.

Discussion:

This made me think a lot of stuff. The first thing that comes to my mind is that we don't know who we are and that we don't know what's around us and that kind of interesting. I mean the information, the computations, and maybe the answers are just out there in the universe and we don't know how to read them. I like the title of the presentation, programming the universe means to me taking the information that the universe has and make it compute.

Every time I see something about quantum computation I can't avoid thinking about cryptography. This makes me realize that this invention can change a lot of things in our lives. Maybe these computers could give us a lot of answers. I will like to know why we are here.

Seth Lloyd: Quantum Machine Learning

Things to learn or note:
Discussion:

The Pretotyping Manifesto

Things to learn or note:

• Law of failure
• Most new ideas fail. (books,apps,etc)
• Make sure you are building the right 'it' before building 'it' right.
• With pretotyping, you're going to have very fast failure and because of that, you're going to have more time to try new things.
• The only things that ideas collect are opinions.
• Don't look for ideas, look for innovations.
• Pretotyping: validating the market appeal and actual usage of a potential new product by simulating its core experience with the smallest investment of time and money.
• MVP: minimum viable product.
• Pretotyping: Is to answer, would we use it?
• The more time you invest in an idea is harder to admit that is a failure.
• Pretotyping is validating that you are building the right thing.
• Pretostorming: Pretotype -> data
• Use data and not opinions
• What you do is create a prototype of what you want to do.
• You create this prototype with the minimum resources you could waste. (money, energy, and time)
• Go out and show it to the people.
• Gather data.
• Use that data to make changes or to make a decision.
• Fail quickly.

Discussion:

I like the idea, we are kind of iterating in the development of the product. I think the most important here is to validate what we are doing, we sometimes think that our ideas are good and well.. sometimes is not true so we need people to tell us that kind of thing. I also notice that you can get quick feedback about your work also if you are going to fail you will do it quickly.

I just have a question. When do I know which is my MVP? What I can understand is the one with the basic functions. What can I also understand from the examples is that you don't even have to develop it, you could show even a picture or a drawing.

Stephen Wolfram: Computing a theory of everything

Things to learn or note:

• Yes, it's true nature builds complex things that to us appear so complex.
• The principle of computational equivalence, which tells us that even incredibly simple systems can do computations as sophisticated as anything.
• WolframAlpha answers questions.
• We can see a lot of examples in this part.
• Wolfram Alpha and Mathematica are now full of algorithms that we discovered by searching the computational universe.

Discussion:

Dude...!! Look what I can do!

This is so cool how they gather data and try to answer questions using simple language.

What I can see here is a different application of computation and there are trying, once again, to answer questions and almost the same questions who we are? how the universe works?. But this guy is talking about difficulties that they encounter, like no having the technology they need and they have to create it. But what I like is that they have a belief and they are working for it.

Seth Lloyd: Quantum Machine Learning

Come on let's learn how Machine learning is going to be with quantum computers.

Things to learn or note:

• Gives a brief introduction to Qbits
• The fact is that this feature that quantum mechanics is about vectors, matrices, linear operations
• To construct quantum machine learning algorithms is to map ordinary machine learning algorithms when they have to do with manipulating vectors, figuring out inner products, performing linear transformations.
• The point here is to transform the data so it can be computed in quantum.
• They assume that all of this stuff can be done at the quantum level.
• You can represent a vector in multiple ways at the quantum level.
• The idea here is that quantum computers work because you're representing all these bits simultaneously.
• Machine learning can be done better in the quantum state because you can find very interesting features.
• In this part of bits and vectors we just need to understand that quantum mechanics allows manipulation of vectors in c^2^n using only n qubits.
• The difficult thing to do here is to get the information in a form where is manipulable.
• Quantum random access, basically takes this information and codes in a quantum digital form.
• To this point, we assume that we have the information in Qbits and the QRAM.
• Now we need to construct overlaps in the vectors.
• Calculate the distance of a vector.
• The quantum versions of these algorithms are exponentially faster than the classical ones.
• Algorithms are faster in vectorial spaces.
• These algorithms are faster because basically, that's what these things do in nature.

Discussion:

What I could understand is that you have to transform first the data to the quantum state in Qbits. At this point, I stopped and try to figure out why do scientists want to do machine learning on quantum computers, then later on in the video, I realized that it was way faster. Now, you have data you need a QRAM that can code information to quantum information. At this point, you can start manipulating data, but some of the algorithms run on their own, or it's easy to apply them, it's all about linear algebra. With these conditions, you can have faster algorithms running.

I can't even imagine how it's the future going to be with all these crazy things improving the way we do things. It seems like very far in time but I wish these kinds of inventions work for good. This was so interesting!

Why you should have your own black box

Things to learn or note:

• The way we conceptualize success shapes the way we behave
• 2 groups some think that talent is relevant the others don't. They have 2 different behaviors.
• A growth mindset recognizes that talent isn't enough and they have a lot of success stories.
• The problem is when you're in the wrong culture, the intellectual and creative energy does not go towards learning but towards self-justification.
• The growth mindset liberates our talent, they tend to learn from errors and be better.

Discussion:

I talked before about growth mindset but in this video, we saw some examples, and what I most take is that talent is not enough, and thinking that talent is enough stagnates us. You have to be able to learn new things, grow, and change. Our errors and understanding those are the way that we can grow.

Richard Feynman

Things to learn or note:

• Feynman invented Feynman diagrams.
• Understanding the world was important to being a person
• Feynman was extrovert and curious.
• Feynman diagrams could be really useful in sorting out the math of QED.
• Feynman "invented" parallel computing.
• Worked in the atomic bomb
• Did a study of the computational limitations due to quantum mechanics.
• The problem of manipulating and controlling things on a small scale.
• Dude! that guy is talking about Feynman and he is crying :c
• "I'll be dead but I won't be entirely gone." :c
• It is scientific only to say what's more likely and less likely and not to be proving all the time.
• With this method, we can disprove any definite theory wrong.
• 1. You have a theory or you are guessing.
• 2. Compute the consequences.
• 3. Compare.
• A theory can't be right if it is not possible to prove it wrong.
• A vague theory can lead to different results.
• We can never be right, we can only prove we're wrong.

Discussion:

First of all, I have to say that I didn't know who was Richard Feynman, and well... I feel guilty for that. What I can say that I learned from this series of videos is that he did a lot of stuff of computing and science, I think is the way he thinks that made him learn and discover a lot of things but I'll get to that, he was great and I'm grateful that I could see these videos.

The other part that I want to talk about is his attitude, the first thing that jumped to me was the fact that he was curious and extrovert, so it seems that he was somebody energetic and he used all the energy on stuff he wanted to learn. One of the things also that I can note is that he was humble, he recognized he couldn't do something(like the computations needed) so this leads him to find new solutions to create new ways of doing things. I think this was one of the points that made him different from other scientists. He had a growth mindset.

The last point is the scientific method that he mentions 3 points: guess, compute and compare. What he thinks and that I take with me is that we can never be right, we can only prove we're wrong.

---

Testing & Automation

Breaking Things at Netflix

Things to learn or note:

• We try to hide our errors and that's why we find errors in production.
• Systems must have a response system in case of failure.
• A system being anti-fragile is something that gets better in the face of change.
• Organizations should take failure and change to get better.
• There is also the FIT framework.
• Failure injections are when you inject or send information that may harm the system.
• The idea is to give people the liberty to do their testing and let them be responsible and take the decision.
• Doing this helps engineers understanding the system.
• People enjoy breaking things.
• Giving them a realistic testing scenario saves time and effort for testing.

Discussion:

I also like breaking things, and I agree with a couple of points breaking things or systems gives you a different perspective on how the system works and how could be improved, and this generates new learning. The other point that I also agree with is that system must have a mechanism in case of failure but that will depend on the nature of the system and needs.

The other part that I like is that organizations are changing and they are giving more freedom to testers and this is great because we can learn and find bugs or failures and we can do this without worrying about budget, I mean is great we can do this.

GTAC 2014: Test coverage at Google

Things to learn or note:

• Code review code compliance, code looks the same and that tests are running
• Tools tell how many tests you should do.
• There are also different tools for tool coverage.
• 85% of coverage
• We need to worry about coverage.
• Test own code, not implemented code.
• Also depends on the language.
• 85 is the magic number

Discussion:

The most important thing that I can note is that they care for testing, it's relevant for google. They are trying to reach 85% of test coverage. So, it means that almost all the code has an automated test. Now what I can conclude from this it's that this improves the quality of the software. The other point it's that tests also work as documentation so if google has an 85% of test coverage then because it's worth it.

I Don't Test Often ... But When I Do, I Test in Production

Things to learn or note:

• Testing is hard.
• Simian Army: It's a set of software monkeys that essentially set failures into a production environment rather than trying to simulate them in a test environment.
• The second method is running code coverage analysis in production.
• I'm going to talk about canary analysis, which is a way of verifying new features and changes, again, in a production environment.
• We're constantly testing multiple features like that in production.
• OMG Netflix tests in production!!! why?
• OMG, and it's so complex the distributed system!
• Security groups that effectively are software firewalls that ensure the connectivity between services is correct.
• Now landing the idea, Netflix is so complex has a lot of instances in AWS, and lots of things.
• One problem is that they are not in charge of the hardware and they can't monitor everything.
• They also had problems with natural disasters.
• They also had problems with human errors.
• The goal is availability.
• Exception handling, redundancy, or degraded experience.
• It's hard to test because of the massive, rapidly changing data, internet-scale traffic, complex interaction, controlled systems, and finally, they are always innovating.
• Chaos monkeys. Simulates failures inherent to running in the cloud and during business hours.
• Chaos gorilla: Simulates unavailability.
• Chaos kong: Simulates unavailability in a region.
• Latency monkey: Simulates degraded instances.
• Conformity monkeys: Apply a set of conformity rules to all instances.
• Janitor, Security, and Howler monkey.
• Run your code coverage analysis in production.
• They are trying to create new failures and modes.

Discussion:

What I can note here is that software engineers and in general, the industry is changing the traditional way that is testing. They are creating new big ways of testing because is necessary to test software from different views. I like the way they are using creativity to break the system and how they let engineers the responsibility of testing and proposing solutions. The main point, in my opinion, is to test do it the way you want and need, but do it.

GTAC 2014: The Testing User Experience

Things to learn or note:

• It's is important to have a good testing culture.
• Testing is a hard problem.
• Requires resources, writing tests it's not that easy, having engineers motivated to do this.
• How we can motivate engineers to write tests. They think it's a waste of time. This can be the problem.
• Sometimes it's a waste of time.
• In general, testing is expensive
• Static analysis
• We can build tools to help us test.
• There are some ways we can improve our testing process.
• You can make sure that when you write a test and you have an assertion and the assertion fails, it prints out a message that's clear about what's going on.
• Build the right tool with the things that the users need.
• Managing a breakage
• If we want our users to be happy, then, like, we should only bother them when we really...
• Find solutions...
• Find the right person
• I think the best way to get engineers excited about testing is to make it not suck for them.

Discussion:

Well, there are a lot of things that I didn't understand, I mean is how google changed the way testing was done and... well their process. Some technical issues and how they solve them... I don't know... maybe in others places works differently right.

What I can see it's that they faced a problem, they had a lot of issues with testing, the main that I can see is that engineers think is a waste of time testing, and yeah sometimes it is if you are adding a simple functionality and well you are going to waste more time testing than coding it. The way and I think is the correct way is to help people understand that testing is not a waste of time and more important... they change the way that test could be done. I think it was a great approach to try to ease a hard problem and this changes the user testing experience.

GTAC 2014: The Testing User Experience

Let's learn about google's integration tools.

Things to learn or note:

• God... they got a very big project
• An integration system does mainly three things.
• Provide real-time information when there is a build break in a branch
• Provide frequent green builds for cutting releases
• Develop safely.
• This is how it works, a new change comes in and more changes can come, then runs all the tests when it's finished says if it passed or failed, and this way we can know that something failed.
• 
• What they do at Google is that they start testing then if a change is made then pick it up when we start testing it
• 
• By doing this makes the build monitor easy.
• What they do is that they quickly can find who broke the code and ask that person to fix it.
• Tests every change, quick response and enables the team to iterate fast.
• It does have its cons, requires lots of investment to compute resources, and requires updating dependencies on the change (the dependency map).
• Also improves safely development.
• Each team has its group of tests.
• The system assumes that test passes or fails reliably given code and flaky test don't have this property, they can fail because of time out.
• Causes: inability to find changes breaking the build.
• Fix it fast!
• They also have metrics to measure the fails and flaky
• Practical matters!

Discussion:

Wow more than a way to test they got a whole culture to testing, I mean they had to change the way they work but having in mind testing. I liked the explanation, it was so technical some things could understand but I could manage to understand the process of how they integrate and then tests runs. I think also that this is very agile I mean if something it's not right the go back and fix it but this has to be quick.

I also like how Google and employees al willing to share that information with us, I mean they are trying to say that testing is important and that is part of their culture and also have metrics and the put types and they know their limitations, I also think that they learn a lot of their errors. Finally, I liked how they do it on google but could have some problems in other areas.

Testing Engineering@Google & The Release Process for Google's Chrome for iOS

Things to learn or note:

• The test engineers and test culture are not QA.
• It's about innovation and automation.
• Product quality and testing is a collective effort.
• They develop build, release and test automation frameworks.
• Testers can show that all the use cases that we can think of are properly validated, implement them into correct automation to support that.
• QA and testing are separated.
• Google develops also IOS apps.
• They use GIT and some automated build solutions.
• Chrome on IOS.
• They are facing another environment where they have more restrictions.
• They have to work to find solutions for those restrictions.
• Release internal-only dev, canary and beta build.
• They have to validate the new changes for Chrome.
• Once approved, they branch the milestone or change to chrome.
• Release-dev process.
• 6 weeks of fixing and testing on the branch.
• Chrome IOS is sent to apple and they take to the time to review it.
• They wait like two weeks... it's a black box process.
• Finally, they will push it to the app store.
• They do automation on testing.
• It's a group of bots that test and give feedback fast.
• You want feedback fast that's why you automate things.
• The bots are unit tests, end to end tests, performance tests, and screenshots tests.
• On the other hand, we have manual testing because not always you can automate everything, flakey tests and sometimes tests haven't been coded yet (technical debt).
• They do a lot of testing in different like security, user type, etc.

Discussion:

I have used a couple of time chrome canary and I didn't know the reason to be "canary". in this video I discovered that it's the result of a channel and here they have different functionalities which are still testing and yes, it offers other things. What I can learn from this is the way that they are testing, just adding new functionalities to other channel or app, and they finally when everything is working properly add it to the main app.

Also what I can notice is that they understand that is a hard process and sometimes they are going to have problems and sometimes the app is going to get rejected. I think this is the main reason to do another app and take the time to test everything and make sure that everything works properly. This a different way to see things regarding testing.

Chaos Engineering: the history, principles, and practice

Let's see what is Chaos Engineering, I did a cheatsheet for this, I'm going to paste it here.

---

---

Three experiments on premise validation using the pretotyping technique

By: Ivan Alberto Aguilar Castillo, Hernán José Cervera Manzanilla and Luis Fernando García.

May 3, 2021.

Cooking App for Everyday Recipes: Suggestions Based on Ingredients You Already Have

Description

We know from experience that most college students or people just beginning to live alone tend to have trouble cooking. The idea of this app is to give users an easy way to check a variety of recipes, furthermore, it would allow users to register their ingredients so that they can search for recipes that they could do with what they have in home. The main objective of this application is to help users with little cooking knowledge to start preparing food and improve their culinary skills.

Hypothesis

People who are curious about learning how to cook would be interested in using an application for finding recipes according to specific ingredients.

Experiment description

The idea of the experiment is giving people a series of drawings that represent the user interface of the system, these drawings will be linked by hyperlinks. After that, the user will answer 3 questions about their experience. The purpose is to validate if they would be interested in using the application.

The process is the following:

• 1. Give user a context of the application.
• 2. Share the link of the pretotype with the users.
• 3. Share the Google forms, with 3 questions that the users will answer.
• From 1 (not very useful) to 5 (very useful), how useful do you think the product would be?
• From 1 (not very interested) to 5 (very interested), how interested would you be in trying out the finished application?
• Would you recommend the working application to someone else? (Supposing it meets your expectations).
• 4. Analyze the results of the experiment.

The message that is sent to the groups is the following:

¿Te interesa aprender a cocinar? Esta es una aplicación para buscar recetas, cuya función principal es la de agregar ingredientes (los que tengas en casa, por ejemplo) y buscar recetas en base a los ingredientes que agregaste. Validation

We will measure interest based on the average obtained from user responses. We will interpret each of the results of the questions based on percentages, where 100% will be the highest interest, and 0% the lowest.

The Google form tool directly provides the percentage of results per question. The hypothesis is accepted if at least two of these considerations are passed:

1. The total of users that think the product would be useful (4) or very useful (5) is equal or higher than 80%.

2. The total of users that would be interested (4) or very interested (5) in trying out the finished application is equal or higher than 80%.

3. 80% or more of the users would recommend the working application to someone else.

Results

The link and the form were shared in two different chat groups where 13 people tested the pretotype and answered the questions. The results are the following:

• 1. A total of 10 users (77%) think that the product would be useful/very useful.
• 2. A total of 11 users (84,6%) would be interested/very interested in trying out the finished application.
• 3. A total of 11 users (84,6%) would recommend the working application to someone else.

Since two of the three considerations were approved, the hypothesis is accepted. We conclude that there are people who would be interested in using an application to search for recipes based on specific ingredients.

Nutrition Meal Plan Recommender

Description

The idea of this app is to get a recommendation of a meal plan based on your lifestyle, weight, height, and other medical data. The twist is that if you are not sticking to your dieting plan, the app asks the user possible reasons why this might be and according to the input the diet plan changes. The main objective is to help users find meals that could keep them healthy.

Hypothesis

People who care about what they eat would be interested in using an application for getting a custom meal plan recommendation based on their lifestyle and medical status.

Experiment description

A post was created that describes the general idea and we are inviting people who are interested to click on a special link that could give them more information about the product. However, this link serves merely as a counter. This is the message we shared along with the link:

App que recomienda planes de alimentación de acuerdo a tu estilo de vida, peso y e información de salud. Si no estás cumpliendo con el plan, la app cambia el plan de acuerdo a por qué le dices que no te está funcionando, p.ej. no te gusta el sabor, mucha preparación, monótono, etc.

The duration of the experiment is of one day.

Validation

We are going to share the link in some groups and we can count the number of visits that the page is receiving. This way we want to measure how many people out of group could be interested in the app.

Having this in mind we could measure the interest of the total group with the following formula:

Interest percentage: 100 * (clicks received / total number of people)

Results

We shared the post in two groups, in the first group there were 22 people, and in the second one, there were 16. In total 38 people could see the post and click the link to get more info. Finally, we could see the number of interested people in the product:

Interest percentage: 100 * (9/ 38) = 23.68%

Since we didn't get to the expected percentage, we can say that the hypothesis is not accepted. People who care about what they eat are not interested in an application for getting custom meal plans. They aren't interested in the product.

All the Nutrients You Need in One Beverage

Description

I (Hernán Cervera) sometimes have a very tight schedule and I realize that eating alone takes almost 3 hours of my day, which I think it is a lot of time that I could invest on other tasks. I believe that other people are in a similar situation, and if cooking is involved, the occupied time might be even more. The idea is to create a beverage which contains all the nutrients that you need for a meal, making eating a matter of drinking something which doesn’t taste awful and your nutritional requirements won’t be compromised.

Hypothesis

People who usually have busy days would be interested in a drinkable meal.

Experiment description

A Facebook (FB) page is created for the fake company Nutrición Concentrada, which has a post about the drinkable meal, a fake product of course. The post contains a short description and a mocked image of the product.

The link to the post, along with a brief message, is shared to chat groups where we believe there might be people with tight schedules:

¿Quisieras invertir menos tiempo comiendo? ¡Este producto de Nutrición Concentrada ofrece los nutrientes de una comida entera en una bebida! En caso de estar interesado apreciamos reacciones a la publicación de FB, buen día 😁

The duration of the experiment is one day.

Validation

We measure interest by the percentage of positive reactions (like, love, care, wow) to the post by the members of all chat groups combined.

Interest = 100 * (positive_reactions / number_of_people)

The hypothesis is accepted if the interest is greater than or equal to 80%.

Results

The link along with the message was shared in two chat groups with total 30 people. The FB post got a total of 5 reactions of the type “like”, which corresponds to a positive reaction.

Interest = 100 * (5 / 30) = 16.66%

Since the percentage rate is lower than 80% (an in fact, much lower), the hypothesis is not accepted. We conclude that people who usually have busy days would not be interested in the product.