Tech growth in Armenia, state of the industry

in

The tech industry in Armenia is generating a lot of hype but much of it is written from the non-technical perspective so its mostly fluff and smoke. This short post is intended as a overview of the state of the industry in 2017, what technologies are hot, and what kinds of things you can expect if you move here.

NOTE: Usual disclaimers, this is my point of view, intuitions, etc etc

Technologies and Getting work

Armenia is surprisingly ahead of our many neighbors in the tech stack of many businesses, for example compared to Lebanon we are more advanced. There’s still a lot of PHP and Java jobs but Armenia is picking up in using more modern and hot technologies like React, ReactNative and nodejs (I’d like to think I had something to do with that). Startups are starting to use those cooler technologies, older boring companies will be using the usual boring, soul-sucking enterprise tech.

Now if you are talented and can demonstrate that preferably with public code on github, then you can get a job in 10 minutes so in that respect its like the Bay Area. Most of the companies aren’t large enough to justify having positions like Project Manager so programmers will have an easier time getting a job. If you can’t get a job as a programmer then it means you aren’t good enough yet so take it as an opportunity to improve yourself and learn something more relevant.

There’s also some C++ work but you’ll probably only want to deal with the good C++ places, aka the game developer studios; they are pretty good and at least know what C++11 is.

Problems

The industry does suffer from a lack of experienced, ‘senior’ talent. The tech industry is especially subject to the problems of emigration as people are easily poached once they reach mid-level experience and technical ability. In my opinion what passes for ‘senior’ level person in Armenia is really mid-level or less in the Bay Area and its not because the people are stupid but mostly because the people with the greatest potential physically up and leave the country. This has massive ripple effects as that knowledge is never passed down to others and companies cannot expand their workforce or quality of their code.

The plus side is that if you’re an programmer with real industry experience then you have a big chance to make an impact in a company that would take a bit longer than back home.

Places to hang out

The country is starting to have regular tech events, so you can expect at least two cool tech events each week. These will be mostly at ISTC, iterate hackerspace and at MICArmenia

Practical Armenia, my experience - Part two

in

This is part two from the my first post.

On mobile might need to scroll the pro-tip/anecdotes horizontally.

Financials

Once in Armenia you’ll need to get a feel for the prices and whatnot. You should need to spend at most 5,000 drams a day, that’s like $11.

PRO-TIP: $11 is a slice of pizza and beverage in San Francisco. 

I can easily survive on just about 1,500 drams a day and I eat much better than I ever did in America.

Rent should be about at most $300 a month, which I think is quite fancy and in the city. I spent $250 a month last year during my first stay in Armenia in a very close to the center, Kentron, part of Yerevan, keep in mind that outside Kentron the prices are much cheaper.

You should also make an Armenian bank account, the Armenian financial system is surprisingly strong, although I suspect at the cost of massive government debt, and I like using AmeriaBank as they have a pretty good online banking experience and wire transfers from my American Chase account only take three days. The local branches are super-efficient and everyone knows Russian, probably English too.

More cultural observations

If you don’t know Armenian, then be prepared to answer why you don’t know Armenian. Everyone will ask why you don’t know.

PRO-TIP: Its a legitmate question, why don't you know Armenian?

Sign up for an Armenian class ASAP, I really recommend ArmScoop’s class Armenian as a foreign language as its intensive, fast paced and immersive.

ANECDOTE: I was outside my Yerevan apartment last
year and an older man started asking me something in Armenian. I
replied in Russian that I didn't know Armenian yet, he then asks
'okay, well where is the local police station?'. I told him I didn't
know that either, to which he said: "You don't know Armenian, you
don't know where the police station is, what are you good for
anyway?"...to which I told him I'm an American Armenian and I moved
here. To that he replied, 'Ah well, good job son, you're a good
person'.😄

You are in a different society

Society in general is quite a bit more conservative, but that doesn’t mean you can’t walk around with a Mohawk (at least in Yerevan you could)

that said…

ANECDOTE: I was walking to the metro station on a street 
filled with shopkeepers. In front of me was a mother 
with a child in her arm, kid was on verge of tears but
the mother's soothing kept him just on the 
precipice. As we walked down the street literally every 
shopkeeper pitched in to help keep the kid calm. That would 
never happen in NYC America or SF America, maybe in the
American Southern states.
PRO-TIP: This is a preferable society.

You really feel more connected to other humans and especially as a Diasporan coming to Armenia proper as you’ve made a conscious effort to change the course of this society.

PRO-TIP: You will feel very good when you see the 
fruits of your labor.

The only culture shock

Maybe because I grew up in a Russian environment, that is Russian at home and being a Brooklyn kid, I didn’t have all that much culture shock other than gender relations here which are quite a bit more conservative than San Francisco, for example, but then again which society isn’t? My biggest culture mismatch mostly related to dating as it’s pretty hard to get women to physically meet you and sex before marriage is mostly a myth, almost a double tragedy is that the girls here are much prettier than in America and dress on average much better.

PRO-TIP: Your intuitions will be busted, need to adjust.

Effort

So not everything is 100% ideal in Armenia, but for the things I’ve lost there’s a lot that I gained.

Practical Armenia, my experience - Part one

in

Motivations

My friend Vartan Marashlyan from Repat Armenia asked me to write some blog posts about what I thought would be practical to the Professional Armenian diaspora writ large. I’m writing this in the hopes that it will dispel nonsensical, toxic myths about Armenia and inspire many more talented young professionals in the Armenian worldwide Diaspora to give Armenia a chance.

Keep in mind that this is my point of view

See part two here

On mobile might need to scroll the pro-tip/anecdotes horizontally.

About me

I don’t want to waste too much time talking about myself so I will keep it brief. My name is Edgar Aroutiounian, I’m a 27-year-old Armenian-American and I was born in T’bilisi. In 1993 my family immigrated to the United States and I was raised in New York City then spent teenage years in Florida, started adult life back in NYC and finally moving to San Francisco for a neat career opportunity.

Growing up I spoke Russian at home, no Armenian, I was never taught it. Career wise I am a polyglot programmer, which means I work comfortably in multiple domains, programming languages. Basically, if it involves programming, I can do it.

During Mid-2016 I decided to drop everything and move to Armenia.

Going to Armenia, the build up

In late 2015 I read My Brother's Road, a book about one of my childhood heroes, Monte Melkonian. The book started something in me that I didn’t realize at the time would push me over the Atlantic. At the time I working at a San Francisco startup, living in Oakland; my job was to reverse engineer iOS. By April 2016 I had left that startup and was being courted by Facebook, and Google. Funny story, I have hung up Google twice out of nervousness during the second stage programmer phone interview (I choked).

April 2016 was of course the Four-day war in Artsakh when Azerbaijan attacked our homeland, that bothered me greatly. Around that time my personal life was falling apart as well, I lost my closest person, and that was entirely my fault; I am still recovering from that. Then in mid July the Sasna Tsrer crisis happened and I read what the war heroes said, I realized that the previous generation had given up so much to establish Armenia & Artsakh as independent entities but now the State was faltering, stagnating and going nowhere fast. Our enemies also knew this and so did the native population, (30% of the Armenian native population has physically left Armenia). So I decided to stop everything and go to Armenia, I decided that on a Tuesday, told my then girlfriend that I would pay for her airplane ticket to come with me. By Thursday I had bought the tickets and by that following Sunday I was in Yerevan, that was August 1st 2016.

(Actually, I didn’t even have housing before landing, I secured it on AirBnb right in front of the gate as the connection flight from Doha to Yerevan was boarding).

Landing in Armenia

I don’t have any blood relatives in Armenia so no one met me at the airport and I was subsequently ripped off by my first taxi driver in the country, I didn’t have an intuition yet about dram values. My first place, the AirBnb, was in Erebuni and was decent. I had always read that the Caucasus region was known for its hospitality so I figured I could basically wing it, and by ‘it’ I mean literally everything. On day three in the country I attended a movie screening at AUA, the American University of Armenia, and after the movie was over I announced who I was and what I was doing. Truth be told I didn’t know what I was doing but I did say that I was a programmer from San Francisco and I wanted to help the country become better in whatever way I could. Afterward some students came up to me and I started making friends, one of whom later got me a hookup for renting an Apartment from his extended family. With housing secured I could focus on work.

PRO-TIP: You need to be talking to as 
many people as possible but also
filter out the negative people.
MISCONCEPTION: Armenia/Yerevan 
is a dangerous 
place where you have to
pay mafia people for 
everything and anything.

REALITY: Completely bullshit. 
Yerevan is incredibly safe and I never
had to deal with any shady people. 
Oakland, CA is more dangerous than
Yerevan, Yerevan doesn't have LA style 
drive bys or NYC style random beatings.

Establishing myself

Up to that time I didn’t have Facebook, ironic as I was interviewing with them in Menlo Park, CA, but I soon realized that Armenia lives on Facebook so I had to make an account. I started talking to many people, as many as I could and that meant meeting many people in coffee shops, basically two people a day for about three weeks.

PRO-TIP: Respect yourself and worth. 
Many people will see you as a
goldmine or bag of money. Respectfully 
listen and say you will get
back with an answer, 
always be professional.

Initially I set up roots in Impact Hub, but left within two weeks after I realized that their visions did not align with mine. I have strong opinions and I think that NGOs in general have failed Armenia and that business, profit making, employing people is the smartest way to build the country up ASAP. Being a programmer, I started looking for more serious directly involved tech people and started thinking about what I was missing.

PRO-TIP: Don't waste any time if 
visions don't align, move on.

Around the end of August, I came up with the idea of a hackerspace, a place where programmers could meet up and exchange ideas, work on projects together. Nothing of the sort existed at the time and I started networking to find willing hosts.

PRO-TIP: You need to know more than 
just English to get anything
serious done, Russian does great.

I started with the usual suspects, AUA, TUMO, etc. but was greeted with basically the same replies of “Oh that’s a great idea” but when push came to shove, aka providing a physical place & computers, then everyone became suddenly shy. TUMO especially surprised me in their rejection as at the time the entire upper two floors were completely empty.

PRO-TIP: Never get discouraged.

I also met with many Armenian local tech companies, but then backed out of relationships with some of them once I realized that they wanted me to be effectively a free trainer for their workers, teaching specifically, their technologies and not the ideological freedom that I needed.

PRO-TIP: Cultural pressure to do what 
people what you to do is much
more powerful in Armenia than in America.

Eventually I met the right people at the Innovative Solution & Technologies Center, ISTC, and they took a chance on the idea, the very day of being given the green light I started the first coding working shops.

REALITY: Any diaspora Armenian doing useful 
things in Armenia will be
warmly greeted, accepted. People know its 
odd for someone to leave
wealth for Armenia, you will be appreciated 
especially if you're from
tech since its basically the most important and 
growing industry. I
have never NOT ONCE ever felt like a 
stranger in Armenia, always loved.

Tech scene, resources

MISCONCEPTION: Armenia is broke, 
nothing works, everything is old.

REALITY: My internet speed in Yerevan 
was faster that what I got in
Silicon Valley. Physical infrastructure 
is pretty good, the metro
system is limited to basically the center 
of Yerevan but its cheap,
like 20 cents a ride. People have the latest 
iPhones, iMacs...lack of
physical tech is not a problem.

The tech scene is doing well and everyone is in a way pinning their hopes on it, however what’s lacking now are people with real industry experience who are also willing to share that knowledge.

PRO-TIP: You need to always be 
maintaining your relationships,
otherwise they decaying.

Professionalism

Armenia is still developing in this aspect, so expect everyone to be late at least 15 minutes to everything. The general pace of things is a noticeably slower than what a New Yorker or SF techie might be used to, example: it’s rare for people to walk up the escalator or to use the left side as passing…which for a New Yorker like me…is maddening. Although I am happy to not have to do the PATH WTC train commute, that sucked.

PRO-TIP: Don't lower your standards but 
be cognizant of cultural diffferences.

This is it for now, tweet me at for feedback Էդգար.

Hacks, cracks, dev culture and Armenia's development in tech 🇦🇲

in

Audience


This post is intended for both technical and non-technical people; its for people interested in the development of programmers, cultures of professionalism, security, and the development of Armenia as a tech power. Along the way I will show some security vulnerabilities of well-known sites in Armenia, non-technical people will be able to follow along as everything will be explained. I will also introduce the concept of a bug bounty exchange.

A glossary of technical terms follows the post, might be worth while to read through those terms first.

I want to emphasis that the point of this blog post is not to shame or make fun of any individual or company, rather it is meant to bring attention to the tech industry, culture in Armenia and what can be done to further it to a higher average level of excellence and professionalism.

Problem


Specifically, with respect to Armenia, much has been said about the tech as a shining example of an honest and rapidly growing industry. However, in my view much of that has been pushed by people who frankly are little connected to the tech industry writ tech itself. There are not enough voices heard from the down in the trenches folks who can provide a truly informed voice.

Armenia’s geopolitical situation being what it is makes the tech industry arguably of high interest to national security as:

  1. A robust engine of sustained, organic economic growth. A nation cannot wage wars or devote high levels of resources to its military without a resilient economy. A strong economy lets us negotiate with our enemies from a position of strength.
  2. As a nursery and pipeline for cyber-security. Armenians are disproportionality effective in every domain, the digital realm amplifies that 100X.

For the development of countries, it is better to see a country of makers and not outsourcers. Why? Because making products, like Uber, Facebook, etc, is what makes the biggest bucks of all and is best able to utilize the Armenian people’s creative talents. Working as outsourcing tends to make decent but soulless products, and such products are made by programmers who simply don’t care about what they’re making beyond the salary paid. (Hardly eudaimonia)

With that, Armenia has a lot of outsourcing but the product scene is coming along nicely although we haven’t even seen our first largish failure yet.

Here are two intertwined things I see in the Armenian tech scene that need more attention & sustained improvement.

Problem Fixable Solution
Hacker, developer culture lagging Yes Invest more in people
Cyber-security Yes Invest more in people

What do I mean by developer culture? In a way, it’s like professionalism in other fields which is characterized by devotion to your craft, not cutting corners, doing things the right way even when it takes longer and deeply learning the tools of your craft. Sometimes in Armenian culture people like to cut corners, this is a fact. Examples include cutting corners in Gyumri’s apartment construction which led to worse outcomes in the 1988 earthquake to the elevator in my building whose buttons were always off by one. There’s an implicit acceptance of a shrugging your shoulders quality work and it is reflected in the tech industry as well.

Examples of shortcoming


Now let’s look at a few examples of the manifestations of these shortcomings. I will now show you three examples of live web vulnerabilities of three sites in Armenia, each of which was told that a problem existed but didn’t pursue a fix initially although one has been fixed. All three of them are the result of similar entry level mistakes.

I found out about these exploits after first posting in the Facebook iterate chatroom about bots attacking silicondzor.com

This image is from a server log for silicondzor.com. See the line GET /.git/ HTTP/1.1? That’s a bot checking if it can get our entire git repo, all our source code. After posting that, Sparik Hayrapetyan reached out to me and reported some Armenian sites that were vulnerable to this very exploit!

I verified some of those site and here are how damning it is…

Say we have three websites each ending in a .am TLD:

(the creator field is from attributes given on the public facing sites)

Site Subject matter exploit creator
siteA auction Full git repo, found credentials http://voodoo.pro/en
siteB car rental Full git repo, found credentials sitemax
siteC reading materials Full git repo https://www.studio-one.am

What does it mean to have the full git repo? It means to have the entire history of the source code from beginning to end.

Verification pics:

SiteA – the auction site

First about the programmer culture, this particular code base uses PHP. A programming language known to be inherently defective in security and one that is usally avoided for new projects in Silicon Valley.

Not only are they using PHP, they are also using an IDE to generate PHP code.

Another bad practice they have their passwords in the source code. Hint: this is never a good idea. Since as an auction site they are handling all kinds of credit cards and other sensitive information and now any attacker, including me potentially have access to it all. (Programmers: A better solution is to use environment variables)

SiteA also exposed their SQL on the public website

SiteB – Car rental site

The car rental site required a little bit more work to reconstruct the original git repo. Thankfully using tools like GitTools makes the process painless and after some searching through the source code I found this goodie. These are credentials to the database which being a car rental site also probably contains some nice credit card numbers, accounts.

SiteC – Reading materials

I didn’t dig too deeply in this repo but here’s an example structure. The indentation means hierarchy of directories and files, things ending in .php are source code files.

Studio-One proudly boasts of its clients including AmeriaBank and the National Assembly of RA. How much do you want to bet that they do similar sloppy coding across all these projects?

Again this exploit was rather simple, it was a rookie mistake.

Bug Bounties

Now as I mentioned Sparik first reported exploits to the respective site owners but amazingly some didn’t even reply or simply asked him leave his email address, the de facto equivalent of “Don’t call us, we’ll call you”. Amazingly Sparik wasn’t compensated at all or recognized! I reached out as well but only one replied and has since fixed the mistakes.

Many other companies would have paid him under a system called a bug bounty. This is when companies pay whoever finds exploits on their website/app/program under a structured disclosure method. The idea is that it is better for the company to pay to know about the exploit, fix it, and move on rather than have the exploit end up on the black market and then hit them out in the wild. An example of this is the Target hacking, that cost over $100 Million in damages. Facebook has been running their bug bounty since 2011 with great success, some payouts reach $40,000 which is still substantially less than what can happen when someone malicious literally has all the passwords to your databases and computers. To my knowledge, there are no companies in Armenia that run a bug bounty.

Mitigation

How can we mitigate these kinds of exploits? Well in a way its simple but also hard & vague; we do it by:

  1. Promoting a culture of people open about knowledge, about promoting collerboration. This particular exploit is talked about in just about every other InfoSec meetup in San Francisco and is well known but in Armenia a seemingly prominent firm is repeatedly making it. Practically speaking this means more meetups. A check on silicondzor.com shows 53 events in Feburary for Armenia, that number needs to be higher and a higher percentage needs to be programmers talking to other programmers, not fluff sessions about Marketing/Startups. The meetups should lean toward being workshops with hands on examples and live coding.

  2. Sponsoring bug bounties, especially starting with Government & military websites.

  3. Having more events like the recent Capture the Flag which literally included the exploit used in this blog post.

  4. Promoting a culture of professionalism & respect for programmers. Programmers are not respected as crafts people in Armenia.

  5. Collective funding of talent. The entire Armenian tech industry needs to be willing to spend some money on the collective pool of talent. This means like non-trivial prizes for hackathon, paid bug bounties, paid trainings Armenian culture in general does not promote doing things for free, or helping someone without expecting something in return implicitly.

  6. Promoting and funding projects that protect critical internet infrastructure for Armenia, like CERT-AM

Keep in mind that many of these are happening one way or another, I am merely enumerating some for record. Because I believe in doing and not merely speaking, I will be creating a public bug bounty exchange for all of Armenia, it will be listed on silicondzor.com. Once it is up, I encourage all companies, not just tech companies, to post on the bug bounty exchange with offers of payment for successful examples of exploits.

Parting

(While writing this post a new story broke out of crackers attacking Armenian banks, Hackers Infiltrate Computer System of Bank in Armenia; Steal $273,000, I think you could see the connections of what I’m trying to say; forget criminals for a moment and now imagine a nation-state determined to destroy us.)

There is a lot of hype and excitement for tech in Armenia and it holds a huge promise, but we must grow our industry, talent correctly. In addition, because the topic is Armenia, we must keep in mind about security as cyber-security is increasingly the first line of attack and defense in war. Would our enemies be so kind as to tell us about exploits on our banking sites? Or perhaps military sites or other critical government? What about the electrical system or other critical infrastructure that is increasingly connected to the internet. How effective are Iskanders when someone hacks and reprograms them mid-flight? These questions show the level to which tech, economy and national security are all interrelated in Armenia and the level of responsibility the tech industry has to Armenia.

Glossary

Note:

  1. Non-technical people, you should probably start learning the jargon.
  2. Technical people, these are loose definition meant for intuition rather than accuracy.

source code: The original code of a program, written by a programmer. This can be publicably available or private, usually when its private then it has things like passwords written in it.

exploit: A mistake in the source code which then lets another person use that mistake to cause a program to do something other than originally intended. I.E. if I find an exploit in the ATM machine, then maybe I can get it to spit out money.

InfoSec: Short for information security, its a subfield of programming culture and include a focus on security.

cracker: A professional hacker who attacks programs, products for money usually.

pentest: Short for penetration testing, its when you pay a infosec professional to attack your product as if they were a cracker.

DDoS: This is when someone abuses whatever service you are providing by overwhelming you with too many requests for that service thereby bringing the whole system down.

git: A program used by programmers to help them keep track of all the source code they write, down to the detail of which line of code was written by whom, when.

repo: A directory on the hard drive of a computer created by the git program. It contains all the source code and whatever else the programmers decided to keep track of in the project.

commit: Something created by git that is like a collection of saves of source code. Think of it like a record in time of how the source code look like.

TLD: Top level domain, basically the .com or .am part of a website name.

IDE: A program that some programmers use to help them make new programs, think of it like a fancy word processor with auto completion.

SQL: A programming language designed for databases. If I know your SQL then I know how your data is organized, stored, and what I should be looking for once I get into your system.

bots: Computer programs designed to do boring, repetitive things. In this post it refers to programs designed to search the web for simple, rookie level exploits.

let binding in OCaml class definition

in

I like the object layer in OCaml but here’s one quirk of the language that sometimes I forget about and it can bite you…like I just got bit in my OCaml bindings to Java’s ScriptEngine. (Let’s you evaluate JavaScript in OCaml using the JVM)

OCaml lets you define an object like so:

class thing = object
  method speak = print_endline "Hello"
end

let () = (new thing)#speak

Note that methods don’t need arguments, they will always go off when you call them.

and you can also have fields

class thing = object
  val coder = "coder"
  val mutable name = "Edgar"

  method speak = print_endline (name ^ coder)
  method set_name s = name <- s
end

let () = 
  let p = new thing in
  p#speak;
  p#set_name "Gohar";
  p#speak

Notice that we can also make fields mutable and they are private by default.

Now here’s one situation you might encounter:

class compute = object
  val first_field = Other_module.init ()
  val second_field = Some_module.use_it first_field
end

This won’t work though because you can’t use one field in another field.

One solution might be:


class compute = 

  let first_field = Other_module.init () in
  let second_field = Some_module.use_it first_field in
  object
    val first = first_field
    val second = second_field
  end

Now question, are first_field and second_field created each time a new instance of compute is made?

The answer is no and this might be counter intuitive to some, at least it was to me and I sometimes forget this.

Verify it with:

class thing =
  let foo = 1 + 2 in
  let () = print_endline "I was called" in
  object

  end


let () =
  let a = new thing in
  let b = new thing in
  ()

And see how many times I was called is printed to the screen; hence be aware of this when you use objects in OCaml.